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We study mean field systems whose free energy landscape is dominated by marginally stable 
states. We review and develop various techniques to describe such states, elucidating their physical 
meaning and the interrelation between them. In particular, we give a physical interpretation of 
the two-group replica symmetry breaking scheme and confirm it by establishing the relation to the 
cavity method and to the counting of solutions of the Thouless- Anderson-Palmer equations. We 
show how these methods all incorporate the presence of a soft mode in the free energy landscape 
and interpret the occurring order parameter functions in terms of correlations between the soft 
mode and the local magnetizations. The general formalism is applied to the prototypical case of 
the Sherrington-Kirkpatrick-model where we re-examine the physical properties of marginal states 
under a new perspective. 



PACS numbers; 75.10.Nr,11.30.Pb,64.60.Cn 
I. INTRODUCTION 

Disordered, frustrated systems often possess a multi- 
tude of nearly degenerate metastable states which are 
at the basis of glassy phenomena like slow relaxation 
to equilibrium and aging. In order to understand the 
dynamics of such systems it is important to character- 
ize their corrugated free energy landscape, as well as 
the physical properties and abundance of metastable 
states, usually described by the complexity function (also 
called configurational entropy in the context of amor- 
phous solids). 

In finite-dimensional, short-range interacting systems, 
it is not possible to rigorously define metastable states 
since nucleation phenomena always restore ergodicity on 
sufficiently long time scales. However, in mean-field mod- 
els where an analytical description of the free energy 
landscape in terms of local order parameters (magneti- 
zations) is available, metastable states can be identified 
as stable stationary points of the free energy functional 
(see e.g. Ref. for an instructive review of the state of 
the art). The properties of the local neighborhood of the 
latter allow for a natural classification: (i) genuine min- 
ima (with a positive definite free energy Hessian),^ (ii) 
marginal states (with eigenvalues of the Hessian tending 
to zero in the thermodynamic limit) (Siii^ 

Recent studies of the Ising p-spin model at low temper- 
ature have lead to the following picture for the metastable 
states as a function of their free energy density: at low 
free energy densities most metastable states are gen- 
uine minima, while above some threshold free energy 
entropy is dominated by marginal states with a sin- 
gle soft modeM The same phenomenology is found in 
mixed spherical s-|-p-spin models.'S^ In the special case of 
the spherical p-spin model, ^'^^ the dominant metastable 
states are genuine minima at all free energy densities up 
to the threshold of dynamic arrest, above which there are 



no stable states anymore^ 

In certain models, however, marginal states are ex- 
ponentially more numerous than genuine minima at all 
free energy densities. This situation is expected to oc- 
cur generically in models whose thermodynamics is de- 
scribed by continuous replica symmetry breaking, the 
best studied case being the Sherrington-Kirkpatrick (SK) 
modeli^i^iii 

The studies of the above models have shown that very 
often marginal states possess only one single soft mode. 
Such states require a special treatment both in replica 
and cavity approaches, and we will focus on their descrip- 
tion in the present study. Under certain circumstances, 
however, marginal states with a large number (diverg- 
ing in the thermodynamic limit) of soft directions in free 
energy landscape may occur, the best known example 
being the thermodynamically dominant state of the SK 
model below the critical temperature Tci^ This global 
free energy minimum is a stationary point with many al- 
most flat directions in free energy landscape. Since there 
is a multitude of marginal directions none of them can 
be singled out a priori. It is thus not clear whether the 
methods presented here, tailored to the presence of a sin- 
gle marginal mode, may still apply to that situation^ 

The dynamical behavior of a given model will strongly 
depend on the local environment of the metastable states 
that dominate the landscape in the range of dynam- 
ically accessible free energies (usually energies where 
marginal state dominate). To date, the role of higher- 
lying marginal states in the slow dynamics of mean field 
glasses remains unclear. However, the characterization of 
their local properties presented here should help to iden- 
tify their traces in future analytical or numerical studies 
of glassy dynamics. 

In this paper, we focus on the analytical description of 
marginal states with a single soft mode and show what 
physical information is contained in three choice meth- 
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ods to describe them: the two group rephca formaUsm, 
the generahzed cavity method and the counting of the 
solutions of the Thouless- Anderson-Palmer (TAP) mean- 
field equations. In particular, we show that the emerging 
order parameters contain information on the correlation 
between the marginal mode and the local magnetizations. 
Further, we discuss the computation of the distribution 
of (frozen) local fields and infer further key characteris- 
tics of the local environment of marginal states, such as 
the spectrum of the free energy Hessian. 

From a detailed analysis of the SK model, it 
will become clear that the compact two-group replica 
methodi^ii^ is an effective means to describe marginal 
metastable states in mean field systems. We compare this 
approach with other methods and exhibit their equiva- 
lence at the level of the annealed approximation. 

The paper is organized as follows: in Sec. ^ we briefly 
review various approaches to stable and marginal states 
and summarize the current knowledge. In Sec. IIIII we 
introduce an exactly solvable toy model whose physics 
is dominated by marginal states. By analyzing it with 
the help of a simplified two-group Ansatz we gain an 
understanding of the physical meaning of the formalism 
that appears in the more complicated mean field mod- 
els studied in Sec. IIVI In Sec. we relate the two 
group calculation to the direct counting of solutions of 
the TAP equations. Rederiving the TAP-complexity fol- 
lowing Bray and Moore, we exhibit the equivalence with 
the two group approach. In Scc. lVII we review and extend 
the cavity method adapted to marginal states. We show 
its equivalence with the two group formalism and confirm 
the interpretation of the order parameters. In Sec. IVIII 
we build the formalism for a quenched two-group compu- 
tation and discuss its physical content, in particular the 
distribution of local fields and soft mode components. 
In Sec. IVIIII we recall the criteria for internal thermo- 
dynamic consistency and local stability, and discuss pos- 
sible scenarios for the evolution of the local properties 
of metastable states as their free energy decreases. In 
Sec. llXI we analyze the local field distribution in the un- 
correlated, high energy, regime at low temperatures, and 
speculate on its consequences for the dynamics. Finally, 
in Sec. 13 we discuss various open questions and possible 
future extensions of the presented methods, concluding 
with a brief summary in Sec. IXII 

II. CLASSIFICATION OF METASTABLE 
STATES 

The choice techniques to investigate metastable states 
in mean field systems with quenched disorder are 

• the replica method with an ultrametric Replica 
Symmetry Breaking (RSB) Parisi Ansatzfi& 

• the direct counting of the solutions of the Thouless- 
Anderson-Palmer (TAP) equations, i^sl^ and 

• the cavity methodJ^ 



All these approaches can be used to describe stable states 
and to compute their properties as well as their complex- 
ity, i.e., the logarithm of the number of states at given 
free energy density. 

However, in situations where the most numerous 
states are marginal, these techniques need to be 
generalized i^SiSi As explained in the introduction, this 
generally occurs at sufficiently high free energy densities, 
and in certain models even at all free energies (in the SK 
model below Tc, for instance). The characteristics of the 
three equivalent approaches to stable states, and their 
generalizations adapted to marginal states are summa- 
rized in table ITTl 





Minima | Marginal 


Replica 


Parisi RSB 
Ansatz 


Two group+RSB 
Ansatz 


Cavity 


States robust 
to addition of a spin 
Single cavity field 


Fragile pairs of 
saddles and minima 

Additional field 
related to soft mode 


Counting of 
TAP solutions 


Saddle point 
of TAP action 
conserves 
fermionic symmetry 


Saddle point 
of TAP action 
breaks 
fermionic symmetry 



TABLE I: Table summarizing the methods to study properties 
of metastable states in mean field glasses. The second column 
refers to genuine minima, while the third column describes the 
necessary generalization in the case of marginal states with a 
single soft mode. 



A. Prevalence of genuine minima 

At free energy densities where the vast majority of 
metastable states are minima of the free energy landscape 
any thermodynamic function (including the complexity) 
is correctly described by Parisi's RSB Ansatz, by the 
standard cavity method or by counting stable solutions 
of the TAP equations (imposing a saddle point which 
preserves a fermionic symmetry of the problem^'^'^'*). 

Mean field models with an ultrametric organization of 
states in two levels (a "one step" structure) usually ex- 
hibit this kind of landscape at low enough free energies. 
Their static properties can be computed either using a 
one step RSB Ansatz^^ or the cavity method includ- 
ing cluster correlations via the so-called "reweighting" . — 
Whenever the one step RSB solution is stable the com- 
plexity of stable minima can be calculated using Monas- 
son's method as the Legendre transform of the free energy 
of m coupled clones 

It is a non-trivial fact that the same result can be ob- 
tained by counting the number of solutions of the TAP 
equationsiSiSi which requires the saddle point extrem- 
ization of a certain action functional. Under the as- 
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sumption that every solution represents a stable mini- 
mum, the corresponding saddle point has to preserve the 
fermionic Becchi-Rouet-Stora-Tyutin (BRST) symmetry 
of the action^^ which generically occurs in the description 
of stochastic equations 



B. Prevalence of marginal states 

In those cases where marginal states dominate (being 
exponentially more numerous than genuine minima) the 
approaches mentioned above fail because they assume the 
states to be stable minima. However, all approaches can 
be suitably generalized to deal with the marginal case, 
too. The purpose of this paper is to examine the physi- 
cal meaning of these generalizations, namely (i) the ex- 
tension of the RSB scheme to a two-group Ansatzji^iiSiSS 
(ii) the inclusion of an extra auxiliary field in the cavity 
method^SiSi and (iii) the counting of TAP solutions via a 
saddle point that breaks the BRST symmetryAi^ While 
some aspects of the generalized cavity method and the 
breaking of the BRST symmetry have been interpreted 
previously in the literature, the meaning of the two group 
Ansatz, as well as the relation between the above meth- 
ods have not been established. The present paper tries 
to fill this gap. Furthermore, we will show how to ex- 
tract so far hidden information from the formalism and 
give the interpretation of the emerging order parameter 
functions. 

Before applying the two group approach to the SK 
model, we shall introduce a simple, exactly solvable toy 
model which illustrates the basic features of the marginal 
states and their appearance within a replica language. 
This will be a conceptual guide to the physics discussed 
later in the context of the more complicated SK model. 



III. A SIMPLE MODEL TO UNDERSTAND 
THE TWO GROUP ANSATZ AND MARGINAL 
STATES 

The two group Ansatz was first introduced by Bray 
and Moore in 1978, Ref. in an attempt to resolve 

the instability of the replica symmetric solution found by 
Sherrington and Kirkpatrickii in the mean-field approx- 
imation of the Edwards- Anderson modelf22r Even though 
it did not turn out to be the correct way to describe the 
equilibrium (the stable equilibrium solution with ultra- 
metric replica symmetry breaking was found soon after 
by Parisi^^), this two group Ansatz endowed with an ul- 
trametric structure can be used to study the free energy 
landscape above the thermodynamic ground state. 

This replica symmetry breaking scheme consists in di- 
viding m replicas in two non-equivalent groups of m + K 
(group and —K (group "— ") elements, respectively, 
and computing the corresponding "replicated free en- 
ergy" . A subsequent Legendre transform with respect 
to the total number m of replicas-^ was found to repro- 



duce Bray and Moore's calculation of the TAP complex- 
it-- 

A very similar scheme of replica symmetry breaking 
with groups of A" — > ±00 replicas occurred in the mean 
field description of the random field Ising model, — 
where "instantons" with this kind of two-group structure 
were used to identify a certain class of Griffith-like sin- 
gularities of the free energyji^. the physics of which will 
become clear from the following toy modeli^ 



The meaning of the two group limit in a toy 
model 



In this section we examine an exactly solvable zero- 
dimensional model which contains the basic ingredients 
to an understanding of the two group Ansatz. 

From the replica solution of this toy model we will infer 
that the two groups should be interpreted as represent- 
ing a minimum and a saddle point that merge into a sin- 
gle marginally stable configuration in the limit AT — > 00. 
Furthermore, this picture will allow us to obtain a phys- 
ical interpretation of the order parameters appearing in 
the two-group Ansatz, as we will later confirm using the 
equivalence with the physically more intuitive cavity ap- 
proach. 

Let us consider the simple model of a particle in a po- 
tential V{(j)) and subject to a random field h with prob- 
ability distribution P{h), 



n{(j)]h)^V{(l))~h(j). 



(1) 



This can be considered as the 0-dimensional case of the 
problem of pinned manifolds, such as the Random Field 
or Random Temperature Ising Model where a very simi- 
lar analysis leads to the description of Griffith singulari- 
ties.^ 

We consider potentials such that for typical fields h the 
Hamiltonian 7i possesses only one minimum at (f>i{h). A 
secondary minimum occurs at (j)u{h) only for rare fields 
that are larger than a critical value h > he- A simple 
example for such a potential is given by 



V{4>) = K 



(2) 



subject to the constraint </) > 0. In this case, we have 
(f)i{h) = for all random fields of practical interest, and 
for h > he there is a secondary minimum <f>n{h). The 
latter becomes marginal as —> /ic, while (j)n approaches 
4>ii{hc) = 4>c- 

At sufficiently low temperature (T ^ V{(t>c) — ^(0)) 
the disorder average of the free energy behaves as 



F 



-p-HnZ = -[3-^ I dhP{h)\n 



-f3H{4',h) 



V{0)+ / dhP{h)e-^'^^^'^"^^y^^^-^^^'^^y^^^\ (3) 
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FIG. 1: Hamiltonian = V{(l>) - h<j) with V{(j)) from 

Eq. 0, plotted for h/ha = 0,0.9, 1, 1.1 and 1.5 (from top to 
bottom). The secondary minimum 0// and the saddle 0max 
merge for h he- 



where we expanded the logarithm and made a saddle 
point approximation to obtain the second term describing 
the Griffith contribution from the secondary minimum in 
rare random fields. 

For simplicity, we will carry out explicit calculations 
for the case of Gaussian random fields with distribution 



P{h) 



exp 



'2hl 



(4) 



In this case, the integral in Eq. (O is dominated by its 
lower boundary for T >T* = (f)chl/hc, 



F w V{0) + const. X Pihc)e 



(5) 



where AEc is the energy difference between primary 
and secondary minimum, evaluated at the critical field 
strength, he, 



cated and averaged partition function reads 

n 

[]d0,exp[-/3F({0J)] 



(7) 



a=l 



t>a exp 



For arbitrary distributions P{h), characterized by their 
cumulants Cr = {h^)c, the replica free energy functional 
generalizes to 



a=l 



r=l ' \ a=l / 



(8) 



Following the recipe by Dotsenko and Mezard'^^, we ap- 
proximate the integral in Eq. Q by determining all stable 
saddle points of the free energy functional F{{(j)a}), and 
summing their respective Boltzmann weights, 

-/3F^h^=lim?^«^Si_l + ^. (9) 

"-♦0 n n n 

The partition function Zj^g = exp[— /3i^j^g] denotes the 
contribution of the replica symmetric saddle point with 
Frs = F({0a = '/'Rs}) = nf-Rs, and Zrsb is the sum 
over all saddle points breaking the replica symmetry. As 
we will see shortly, the latter always comes with a com- 
binatorial factor proportional to n, which cancels the de- 
nominator. 

Let us show that this recipe allows us to rederive Eq. 
(0). We make the most general Ansatz for the config- 
uration {(pa] of a saddle point, collecting the 0's with 
identical values into M groups labeled by i = 1, . . . , M, 
each with ki replicas, i.e., 

4>a = 4>i, a = 1, . . . , fci, 

0a = 02, a = fci + 1, . . . , fci 4- fc2. 



(6) 



1. Derivation with vectorial replica symmetry breaking 

It is instructive to rederive this simple result with a 
replica calculation. Dotsenko and Mezard^^ found that 
the exact low temperature partition function of similar 
disordered single particle models could be reproduced by 
summing a class of saddle points that break replica sym- 
metry in a non-standard (i.e., non-ultrametric), "vecto- 
rial" way. The physical content of this recipe will become 
clearer below, where we will show that a generalization 
of their scheme applied to the model EQ. indeed gives 
back the anticipated result of Eq. ||SJ|. 

For a Gaussian field distribution, Eq. the repli- 



ca = 4>M-, a= {Yjf^i^ ki) . . . ,n. 

In the Gaussian case, the corresponding replica free 
energy evaluates to 



M 



M(^, 



(10) 



The saddle point equations with respect to variation 
of di read 



M 



= ^ ki 



Vi, (11) 
(12) 



For a regular potential F (</>), the number M of different 
solutions of Eq. (|ll|l is limited. In particular, for the 



5 



potential in Eq. jSJ we have M = 3 tor > he, M = 
2 for /i0 = he, and M = 1 ioi h^ < h^- The latter 
corresponds to the replica symmetric saddle point with 

h^ = 0,C^a= 0, Frs = V{0). 

Replica symmetry breaking saddle points exist for 
h,p > he, where we label the global minimum, the sec- 
ondary minimum and the local maximum of Ti. by </>/, 
(/>// and 0max, respectively, leaving their dependence on 
h impHcit (cf. Fig. P). In the hmit of n 0, Eq. UH) 
yields the self-consistency equation for hj, 



ki( 



) + A:2 ((/)// - ^max) 



Phi 



(13) 



Notice, that we choose the number of replicas in the 
minima, ki and ^2, to be positive, leaving a negative 
number n — ki — k2 — (fci + ^2) of replicas in 0niax- 
This is necessary to ensure a positive Hessian of the free 
energy functional Eq. 

Let us now consider the saddle point free energy Fk^^k2, 
Eq. H10|l , as a function of ^2 for fixed ki . For not too low 
temperatures T > T* , Fk^^k2 decreases as k^ increases. 
Hence, the most important contribution to the sum over 
saddles comes from large ^2 3> 1. As is evident from 
Eq. this requires </>// and ^max to approach each 

other, and hence, h^ must be nearly critical. More pre- 
cisely, one finds 



h^ 



he 1 
he 1 

mlk'2^ 

V3 / he 

2 \2phl 



k'2 



O 



(14) 
(15) 

(16) 
(17) 



The total Griffith contribution to the free energy re- 
sults from the sum over all RSB saddle points with the 
corresponding multiplicity, 



RSB 



= lim 



SB 



hmi y 



(18) 



/£l>0,fc2>0 



n 

ki,k2 



E 



fcl>0,fc2>0 



fci!fc2! 



In order to recover Eq. (Q, we need to exclude sad- 
dles with ki — Q. This can be understood on physical 
grounds: the saddle point free energies Fk-^^oM inde- 
pendent of the ground state level </>/. The corresponding 
terms are not suppressed by the Boltzmann weight asso- 
ciated with the excitation probability to the state (/>//. In 
other words, these terms don't know about the ground 
state and hence must be discarded for the calculation of 
the Griffith correctioni^ 



We note that for any fixed ki > 0, the sum over fc2 is 
weakly divergent and has to be regularized appropriately. 
Since the sum is dominated by large /c2, we use Eq. l(T7|) 
to approximate the Griffith contribution 



PFrsb 



(-1) 



K-1 



K 



K 

E 

fc2=0 



K 
k2 



-l3{K-k2)AE^ 



E 

k2>0 



(-1) 



K-1 



.K>0 



e ^''0 In 1^1 + 
P{he)e-^^^^. 



K 



(1 



K 



In 2 



1 — 



'0 e 



~f3AE^ 



(19) 



and we recover indeed the Griffith correction, Eq. ||SJ|, 
due to rare secondary minima. 

An analogous analysis can be carried out in the case 
of non-Gaussian distributions P{h). Instead of Eq. H17|l . 
one obtains the saddle point free energy 



Fi 



fci ,k2 



kiAEe- l3-^Se + 0{l/kl). (20) 



where Se ~ \og[P{he)]. More precisely, exp(S'c) is given 
by the saddle point approximation of the integral repre- 
sentation 



Pihe) 



dX 
2^ 



exp 



-iXhe 



ext 

A, 



exp -A./^. + ^Ja: 



(21) 



that is a good approximation provided that he belongs to 
the tail of P(h). The Griffith term is again dominated by 
ki — \ and 00, yielding AFe « P{he) exp[— /JAiJc]. 

In summary, the above toy model describes a simple 
Griffith phase which is dominated by marginal configu- 
rations. This physics is exactly reproduced by a vecto- 
rial replica symmetry breaking that divides the replicas 
into two (infinite) groups describing a coalescing pair of 
a minimum and a saddle point {(pjj and cj)ma,x)- 



B. Generalization to higher dimensions 

To make contact with more complicated models, it is 
instructive to generalize our simple model to a particle in 
d dimensions, cj) G M.'^, subject to Gaussian random fields 



n{(l);h)^V{(t>)-h-4>. 



(22) 



The minima of ^ satisfy Vy(0) = h. The Griffith 
contributions to the quenched free energy are dominated 
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by rare fields he that are just strong enough to admit a 
marginal state 0c = '/'(^c)- The marginality implies the 
presence of a soft mode in the Hessian of H, i.e., 



det 



Hess((/)c 



= det 



= 0. 



(23) 



This condition defines a hypersurface S in the space of 
fields h. The dominating Griffith contribution derives 
from he (z S which maximizes P{hc) exp{— /3[7i((/)c) — 
7i ((/)/)]}. In the replica formalism this condition is con- 
veniently encoded in the saddle point equation for ki — 1 
and fc2 ^ cxD, analogous to Eq. H13II . 



V \og[P{he 



where 



"0 = lim fc2[ 



>II 



(24) 



(25) 



is proportional to the soft mode of the Hessian (computed 
in (j)c), and 0/ is the primary minimum in the presence 
of the field hr. 



C. Stabilizing marginal states 

Marginal states are rather delicate to work with since 
they are very sensitive to perturbations. A way to cir- 
cumvent this problem is to introduce a regularizer fa- 
voring (slightly) stable states, and let it tend to zero at 
the end of the calculation. Such an approach has been 
implemented in Refs. 21 sj] for the SK model and the 
Viana-Bray model. Here, we show explicitly the mecha- 
nism of this procedure for the toy model studied above. 
A similar analysis for the SK model will be presented in 
Sec. EJ (see also App-EJ. 

Let us reconsider the one-dimensional model, Eq. JQ. 
For slightly supercritical random fields, h = h^ + Sh, 
there is a nearly marginal secondary minimum 0// w 
<j)c + (2(5/1/1/3)1/2. In order to lift the marginality of the 
dominating states, we impose a stabilizing "constraint" 
by introducing a weight factor exp[Xx X {(f>)] in the phase 
space average of the free energy. The function X((j)) is 
arbitrary up to the conditions X{0) — (in order not to 
couple to the ground state) and X'{(j)c) ^ 0. Any other 
specifics of this regularizer will disappear in the end. The 
Griffith part of the quenched free energy Eq. lO then 
reads 



Fa 



dh p(/j)e-/3[«('^";'')-«(^^;'')l+^^^(^^). (26) 



The integrand is maximal at the saddle point h* — hc+Sh 
with 



XxX'i^,) P{h,) 

(2V3)i/2 p'{h,y 



(27) 



corresponding to a state (/>// which is a genuine local min- 
imum with a minimal eigenvalue of the energy Hessian 
which is positive 



A„,in = V"{ct^{h*)) 



(28) 



^3 5h 



Ax. 



The linear response to a Hamiltonian perturbation H. 



'H — hxX{(j)) diverges as dcjj/dhx 
is lifted. However, the product 



as the constraint 



z = hm Ax tt;— 

Ax^O ohx 



P{hc) 
P'{hc) 



(29) 



tends to a finite limit. 

The above reasoning holds for any function X provided 
that X'{(l)c) which is necessary to lift the marginality. 
In the higher dimensional case, this generalizes to the 
requirement 



(0c) -^y^O 



(30) 



where ip is the soft mode of the Hessian. 

As in the one-dimensional case, there is a finite limit 
for the combination 



lim Ax-^T— 

Ax^O ohx 



(viog[P(/ic)]-V^) V^. (31) 



The avoidance of marginality by means of a control 
parameter coupled to a generic weight function X will 
be used again in Sec. IVII where we will apply the same 
trick to the SK model. 



D. Discussion and connection to the two group 
Ansatz for mean field models 



The vectorial replica analysis presented in this section 
demonstrates that sending the number of replicas of two 
separate groups to plus and minus infinity, respectively, 
encodes the marginality of the metastable states they 
describe. In particular, we observe that the sum over 
RSB saddles in Eq. Ijigi) is dominated by terms with ki — 
1 and fc2 ^ oo. The latter refiects the fact that the 
leading (Griffith-like) contribution to the free energy is 
due to field realizations in which the secondary minimum 
is just marginal. 

Furthermore, in the d-dimensional case, we have seen 
that the difference vector ip between the secondary min- 
imum (pii and the saddle 0max, corresponds to the di- 
rection of the soft mode in that marginal state. This is 
shown to emerge (in Sec. IIII B|) assuming that the inte- 
gral over the field h is dominated by a small vicinity of a 
single saddle point he where P{h) takes its maximum on 
the marginal surface S defined by Eq. H23|) . 

The contribution to the partition function is domi- 
nated by a single state only as long as the number of 
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minima does not grow exponentially with increasing dis- 
order strength. This assumption breaks down in high di- 
mensional mean field models such as the SK model where 
the dimensionality grows with number of spins d ^ N. 
Nevertheless, in these cases the most numerous states in a 
given quenched realization of random couplings turn out 
to be marginal, even though the origin of marginality is 
different. For any disorder configuration, there are expo- 
nentially many states (solutions of the TAP equations) 
in magnetization space, but stable states are usually less 
abundant than marginal ones since true stability imposes 
extra constraints on the TAP solutions. This argument 
certainly holds if no constraint is imposed on the free 
energy density. However, in many models (e.g., Ising p- 
spin models), there is a range of low free energies where 
typical states are stable, while at higher energies the ma- 
jority of states are marginal. The SK model is special in 
that it does not possess such a low energy regime, so that 
the dominant states at all free energies are marginal. 

The above mechanism for marginality in mean field 
glasses is to be contrasted to the toy model where 
marginality is a consequence of maximizing Griffith con- 
tributions over the rare disorder. Despite this difference, 
the insight from the toy model will help us in the next 
section to obtain a deeper understanding of the physi- 
cal significance of the two group Ansatz for mean field 
glasses. In later sections we will confirm this interpre- 
tation by revisiting the counting of TAP states and the 
cavity approach. 



IV. THE TWO GROUP ANSATZ FOR MEAN 
FIELD SPIN GLASSES 

A. Marginal states in the SK model 

Bray and Moore''^ discovered that their computation 
of the TAP-complexity^^ could be exactly reproduced by 
substituting for each entry in a standard Parisi matrix 
a two-group matrix (with m + K and —K replicas, re- 
spectively), and Legendre transforming the result with 
respect to m, following Monasson's approachi^ Later, 
Parisi and Pottersi^ showed that this equivalence ex- 
tends to the low energy regime of the SK model, where 
full replica symmetry breaking occurs, and also holds in 
a model of random orthogonal matriceSf2S, However, the 
meaning of this remarkable result remained unclear. 

The preceding analysis of our toy model suggests to 
interpret the two group Ansatz as a means to force rep- 
resentatives of the two groups into pairs of almost coa- 
lescing minima and saddles. This picture is strongly sup- 
ported by the recent analytical'' and numerical^*^ find- 
ing that solutions of the TAP equations always appear in 
pairs, one being a local minimum and the other a saddle 
of rank one. The straight connection of such a pair of 
stationary points defines a path in the energy landscape 
that is increasingly fiat with increasing system size. In 
the thermodynamic limit, the Hessian matrix computed 



at the minimum has a zero eigenvalue with a soft mode 
pointing in the direction of the adjacent saddle. 

The only TAP-state without a 'partner state' is the 
paramagnetic TAP solution {rrii — 0} which has to be 
discarded because it is unphysical. The absence of a 
'trivial' ground state constitutes an (inessential) differ- 
ence between the SK model and the toy model consid- 
ered in the previous section: in the SK model there is no 
physically relevant analog of the global minimum (pi of 
Sec. mil The marginal states do not merely occur as high 
energy excitations above some trivial ground state, but 
they are the dominant metastable states at a given free 
energy. Therefore, a vectorial replica symmetry breaking 
with only two groups of replicas (associated to minimum 
and saddle) captures all the information we need. 



B. The order parameters of the two group Ansatz 



In the presence of pairs of minima and saddles, the 
concept of overlaps between different states (similarity 
of their magnetizations) needs to be generalized to cover 
three cases: overlaps between two minima, between two 
saddles, and between a minimum and a saddle. They 
can be described by matrices Q^i^ , Qab~' ^^'^ Qab~' 
spectively. The replica indices a, b indicate the distance 
within an ultrametric Parisi tree of the respective pairs. 

Such order parameters indeed appear in the two group 
Ansatz,^- where one works with two groups of m + K 
and —K replicas. K plays now the same role as k2 for 
the toy model of Sec. lIIII We thus expect that a set of m 
replicas corresponds to a pair of a minimum and a saddle 
whose magnetizations differ by a quantity of the order of 
0{1/K), 



= rrii ± 



dnii 
'2K 



(32) 



where {rrii} is the set of average site-magnetizations of 
the saddle-minimum pair and {5mi\ is the i'th compo- 
nent of the soft mode connecting the minimum to the 
nearby saddle in configuration space. Consequently, we 
expect the overlap matrices to be given by 



Qab — Qab 



— Qab 
= Qab 



Aab 

K 

Aab 

K 

Aab 

K 

Aab 

K 



Cab 

Cab 

Cab 

Cab 

Cab 
2^2' 



(33) 
(34) 

(35) 
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with the following interpretation of the order parameters 



Qab 



An 



Cab 



i=l 
1 ^ 



(36) 
(37) 
(38) 



that are assumed to be self-averaging and only depen- 
dent on the distance between the minimum-saddle pairs 
labeled by a and 6. Note that in particular, Q = Q^^, A = 
Aaa and C = Caa dcscribc the internal overlaps of a sin- 
gle marginal minimum-saddle pair. 

Here we have introduced Qab = Q^b match the 
notation of Ref. [2^ but the difference between Qab and 
Q^^, of order 1/K is immaterial in the two group limit 
where K is sent to infinity. 

In a situation where the dominant states are stable the 
matrix Q is the only order parameter (formally there is no 
soft mode, 5rn = and thus, Aab = Cab = 0). In the case 
of marginal states, the matrix Aab measures the correla- 
tions between the magnetization of a state ({wf }) and 
the direction of the soft mode of another state ({Jm^}) 
at a phase space distance labeled by a and 6 (and vice 
versa). The matrix Cab measures the similarity between 
the soft modes of such states. Note that the picture of 
merging minima and saddles gives a clear interpretation 
only to the direction of the soft mode, while it does not 
determine the magnitude of 5mi. In order to extract 
physical information we should thus normalize the soft 
modes by 6irii — Srrii/ ^ {Smf) = Sirii / -y/ Caa ■ In particu- 
lar, we may infer that Cab/ Caa describes the decreasing 
correlations between soft modes with increasing distance 
of pairs in phase space. Moreover, the angle jab between 
the magnetization vector of and the soft mode in states 
labeled by a,b is given by 



cos(7Qf, 



A. 



{Smf m\) 



(39) 



If we assume that the slow relaxation dynamics follows 
essentially the soft mode of marginal states, we expect 
that the larger jaa the smaller the relative decrease of 
the self-overlap in the course of the dynamics. 



C. SK model: The replicated free energy 

Let us now turn explicitly to the SK model with the 
Hamiltonian 



7Y — SiJijSj, 



(40) 



where Jij is a random Gaussian matrix of zero mean and 
variance 1/iV, coupling all N Ising spins Si together. 



Using Monasson's clone method to access higher-lying 
metastable states, one computes the quenched free en- 
ergy of m copies of the system. 



1 



lim - log ZT' 



ext 
/ 



\ogNj{f) - (3mNf 



(41) 

Z™" is the partition function of n x m copies of a system 
with Hamiltonian (|4()|l . n is the number of replicas intro- 
duced to compute the quenched average, while m is the 
number of real copies (the Legendre conjugate of the free 
energy density, see Eq. (|43l) below). In order to capture 
marginal states, we divide the m replicas further into two 
groups oi m + K and ~K elements each. As in the toy 
model, K is sent to infinity in the end, which forces the 
two groups (at fixed replica index a G {1, . . . ,n}) into 
marginal minimum-saddle pairs. 

In the thermodynamic limit N —^ oo, the left hand side 
of Eq. ||1T|| yields the two group replicated free energy 



-(3m^2G{m) = lim 



N 



lim lim log Z^^ 



while the right hand side evaluates to 



N 



logAfjif) ~ Pmf = !](/) - /3m/, 



(42) 



(43) 



where as usual we assume an exponential growth of the 
number of metastable states with system size, Mj{f) ~ 
exjp[NY.{f)], the latter defining the complexity function 
E(/). From Eq. the function /3to$2g(w) is thus 

seen to be the Legendre transform of the complexity with 
respect to the pair of conjugated variables / and /3to. 

After a standard Hubbard-Stratonovich decoupling, 
one obtains the averaged partition function in terms of 
a functional integral over a nm x nm replica coupling 
matrix Q, 



■DQexp{nNT[Q]}, 



(44) 



ilog^exp(^^^.j.Q-,.^,| (45) 



— - m TrQ 

4 V n 



where indices run through a,b ~ l,...,n and i,j = 
1, . . . , TO, respectively. 

As motivated above, the two group Ansatz consists in 
writing the matrix Q^-'j as n? sub- matrices Qah, each of 
dimension to x m of the form: 



m+K 



Qab 




(46) 



and we adopt the parametrization H33I34|I for the three 
sectors. The diagonal elements are Q"^ = 0. 
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The partition function H44|l can be evaluated through 
a saddle point computation which leads to the thermo- 
dynamic potential 



lim ext J^fQl 

n-.0 Q 



(47) 



lim 



f3' 



+ ^m(l~Q)2-/32^(l-Q) 



f3' 



-mQabAab + —Qlt 



with 



{s^} y at ij j 



(48) 



As for the standard Parisi ("one-group") Ansatz, the 
self-consistency conditions for the large TV saddle point 
read 



lab 



(49) 



where a, a' E {+, — }, and the average is taken over the 
Boltzmann factor in Eq. (|48|l . Here an index j+ is re- 
stricted to [1, 771 + if], while i_ S [m + K +1,111]. Solving 
for Qab, Aab and Cab (cf. Eqs. (|33I34() ') we may also write 



A. 



1 

4 
K 

y 



Cab = X2^(4+-S-)(^ 



(50) 

(51) 
(52) 



which resembles Eqs. (|36I38|I . The detailed connec- 
tion between the two sets of equations is established in 

App. m 

The log-trace term in Eq. H48|l can be re-expressed 
in two equivalent ways which will be helpful to make the 
connection with the generalized cavity approach and the 
counting of TAP states, respectively. Technical details of 
the derivation can be found in App. 1X1 

Following Parisi and Potters^^ one obtains the expres- 
sion 



(53) 



yl0g(l- 777^) 



'-r-rda^ -p-|- drag 

^11 2^i 1, 1 1 (1 - 7772) 

X exp < — \xa tanh^^ 777 

I a 

+/?^ E \^aQabXb + ^maCabmb + TTlaAabXh \ , 
ab ) 

where the sum over (a, h) also includes diagonal terms, 
Qaa = Q, Aaa = A, Caa = C We wiU scc in Sec. IVl that 



this form also emerges from a direct counting of TAP 
solutions. 

In this representation, the self-consistency equations 
(|50|l can be cast into the form 

Qab = {rriamb) , (54) 
Aab + Sabil ~ Qaa) = {rriaixb - Uimb)) , (55) 

Cab-Sab[2Aaa+m{l-Qaa)] (56) 

= {{xa - ■mma){xb - mm,},)) , 
where the average (...) is taken over the measure in 

Eq. insj. 

In appendix we show that the field Xa is in a certain 
sense a twofold Hubbard Stratonovich transformation of 
the spin variables Sa — X^i^a- Its average magnetiza- 
tion is therefore mma, and the terms Xa — mma can be 
thought of as magnetization fluctuations in replica space. 
Indeed this furnishes an intuitive understanding of the 
off-diagonal part (a ^ b) of Eqs. (|54I56|) and supports 
our interpretation of the overlap matrices. The extra di- 
agonal terms (a = b) on the left hand side of Eqs. H55I56|I 
arise due to the fragility of the marginal states, as we will 
see in Sec. IVII from an alternative derivation. 

In appendix^ we derive the equivalent expression 



1 



Vdct M J fj, 27r 



(57) 



exp|^ [mlog2cosh(/?/ia) + tanh(/3/ia)/52:a] 

a 

-^Eil'-^aM}, 



where ^' = (ft.a,Za) and M is the 27i x 2?7, covariance 
matrix 



M 



Q A 
A C 



(58) 



Moreover, introducing magnetization and soft mode vari- 
ables (cf. Eqs. HA17IA18P ). 



?77a = tanh(/?/7a), 

fiZa 



Sma 



cosh {Pha) ' 



(59) 
(60) 



the self-consistency equations (150152(1 take the simple 
form 



Qab = {mafrib) , 
Aab = (rhaSmb) , 
Cab = {6ma6mb) 



(61) 
(62) 
(63) 



where averages are over the measure defined by the inte- 
grand in Eq. (|57|l . We will use this form in the discussion 
of the quenched computation in Sec. IVIII 
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D. Annealed approximation 

In the following we will focus on the annealed approx- 
imation, which corresponds to averaging the two group 
partition function Z™ instead of its logarithm. This is 
known to be an exact description at high enough free 
energy densities, / > P cf. Eq. ^ below. Tech- 
nically, this approximation corresponds to reducing the 
overlap matrix to its diagonal part (a = 6) described by 
{Q, C}, and setting all off-diagonal elements to zero. 

The formalism for a quenched computation with con- 
tinuous RSB is reviewed and physically interpreted in 

Sec. rvnl 

Integrating out Za in the annealed version of Eq. H57|l . 
one finds 



dh 



A 



exp 



e"W(2cosh/3/i)" 

/32 QC - A^ 



(64) 



Q 



/3/itanh jih 



Q 



tanh j3h 



The same result is obtained by integrating out Xa in 
the annealed version of Eq. ^b'A\ and using the relation 
vria = tanh [3ha between magnetization ma and local field 
ha- The integrand on the right hand side has the inter- 
pretation of a probability distribution of local fields (up 
to a normalization): 



-Pa„n(/l) 



j-j: exp|mlog (2 cosh/3/i) — 



2Q 



A 



p QC — A 
—(3h tanh (3h + — — tanh^ 



(65) 



The joint distribution of magnetizations •fh and soft mode 
components Sm can be similarly obtained from normaliz- 
ing the replica-diagonal version of the measure in Eq. H57|) 
and changing variables according to Eqs. H59I6()() . 



In the annealed approximation Eqs. H54I56|I take the 
form (performing the Gaussian averages over x) 



Q = (tanh^ ilSh)) , 

A + 1 - Q ^ ~A~ mQ 



C -2A-m{l-Q) = m 



(/itanh {I3h)) 
2^ h 2mA + — 



(66) 
(67) 

(68) 



1 



A\ (fetanh(/3/i)) 



Q 



the average (. . .) now denoting an integral over Pa,nn{h). 
Eqs. H65I68|I allow one to find the annealed solution 
{Q,A,C} easily, e.g., by iteration. 

The above equations turn out to admit two solutionSfSl 
only one of which (with A,C ^ 0) is physical, as dis- 
cussed further in Sees . IVl and I Villi 

Bray and Moore showed in Ref. [40| that this annealed 
solution is stable with respect to continuous replica sym- 
metry breaking (onset of correlations between typical 



metastable states) as long as / > /* 
m* satisfies the condition 



/(m*) where 



1 = (3{l-Q)+l3{A + 



m*Q 




C + m*A + 



-Q-2A 



(69) 



E. Complexity and direction of the marginal mode 

Evaluating the replicated free energy $2G [Eq. H47|l ] 
within the annealed approximation, one eventually ob- 
tains the complexity as its Legendre transform: 



S(/) = -/3m$2G("i) + "i/3/. 



(70) 



In the annealed approximation the above observables 
read 



dm^2G{m) (log 2 cosh /3 ft,) 



dm 



(71) 



S(m) = /3m" 



(1 - Qf - 2mQ2 _ 4^Q 



dm 



- m (log2cosh/3/i) (72) 



^ [m2Q2 _ 4^(1 _ Q) _ 2(^2 + QC)] , 

that are used to compute the complexity curves of Fig.|21 
In Fig. 13 we plot the angle 7 between the soft mode 
and the magnetization vector in configuration space, 
cf. Eq. (|39|) . versus free energy density for various tem- 
peratures. Only the part of the curves to the right of 




-0.8 -0.75 -0.7 -0.65 -0.6 -0.55 -0.5 -0.45 -0.4 -0.35 
f 

FIG. 2: Complexity curves in the annealed approximation. 
Only the data for / > /* is exact, /max corresponds to the 
states of maximal complexity at a given temperature. The 
high energy part of the curves is not fully shown (they extend 
up to /hbe where E(/hbe = 0).) 
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1.6 
1.55 

1.5 
1.45 

1.4 
1.35 

1.3 I- 
1.25 



71/2 



T=.001 
T=.01 
T=0.1 
T=0.2 



-0.8 -0.75 -0.7 -0.65 -0.6 -0.55 -0.5 -0.45 -0.4 -0.35 
f 

FIG. 3: Angle between soft mode and magnetization of typi- 
cal marginal states. The parameter fo is the value of the free 
energy density at which the complexity of marginal states 
computed in the annealed approximation goes to zero {fo is 
not equal to the true equilibrium value /cq since the approxi- 
mation breaks down below /*). /max denotes the free energy 
density with maximal complexity (see Fig. I^J. 



f*{T) is exact, whereas the low energy part is an an- 
nealed approximation to the regime where states are cor- 
related among each other. Nevertheless, there is a clear 
tendency to increasing towards orthogonality, 7 — > 7r/2, 
both as / is approaching the lower band edge of the / 
interval, and as T — > 0. We will derive this behavior 
explicitly in the low T limit in Sec. IIXI 



We can rewrite this sum in the integral representation^ 
Zj{P;m) J VmVxVTpVip (76) 
exp I /3 ^ 9„i, i^TAP ( { TOi } ) 

i 

-mFTApilm}) y 

One then proceeds by replicating the above expression, 
averaging over the Gaussian bond disorder, decoupling 
quartic terms in x,m,ip,ip by Lagrange multipliers. In- 
tegrating out the fermionic fields and imposing saddle 
point conditions, one obtains the expression^ 



exp|iv[-^a;a tanh"^(ma) -m 

aat ) 

a a 



ab 







{2Aab + mQab)mb 

ah 

+ma{Cab + 2mAab + 'm^Qab)mb) |. 



with 



V. COUNTING TAP STATES: BREAKING OF 
BRST SYMMETRY AND EQUIVALENCE TO 
THE TWO GROUP ANSATZ 

In this section we briefly review the counting of TAP 
states, defined as stationary points (minima) of the TAP 
free energy 

Ftap({toi}) = JjjTOimj + i^/o(g,m,). 



Gab = 2{Al^ + Qab Cab) + 'imAab Q ab + W^Qab 



with 



(73) 



foiq,mi) = - ln(l - mf) + mjtanh ^ m,; (74) 
_ln2-^(l-g)2. 



(78) 



This should be optimized with respect to Qab, Aab, Cab, 
leading to the saddle point equations 

Qab = {rriamb), (79) 

Aab + Sab{^-Qaa) = (Xamb), (80) 
Cab ~Sabi'2Aaa+ 1^(1 -Qaa)) = (XaXb)- (81) 

As in the previous section, these equations admit two 
solutions, i.e., there are two possible saddle points for the 
integrand in Eq. H76I) : (i) one with A — C = 0, conserving 
the BRST symmetry of the action (|7()|) , and (ii) a saddle 
point with A ^ 0, C 7^ which spontaneously breaks 
this symmetry. We will discuss in Sec. lVIIll that only the 
second solution is physical. 



In order to select states at a given free energy den- 
sity we weight the TAP states (labeled by a) with an 
exponential factor: 



Zj{P;m) = 



-m/3FTAp({mQ)} 



(75) 



A. Equivalence with the 2G Ansatz 



Taking the logarithm of Eq. H77|) we obtain 



J^{(3]m) = logZj — lim — logZ7(/3;m) 



(82) 
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where m is the factor in the weight of Eq. (|75() . It takes 
the same role as the total number of replicas in the two 
groups in the previous section. Again, a Legendre trans- 
form with respect to m yields the complexity function. 

Comparing the thermodynamic potentials of Eq. 1)82(1 
and Eq. H42I) (evaluated according to Eq. Hb'3\i ) we estab- 
lish their complete equivalence after shifting the integra- 
tion variable to Xa = Xa — mnia in Eq. (|53|1 . As explained 
in section llVl this shift amounts to using natural variables 
for magnetization fluctuations (see also appendix . 

From the present derivation we obtain a better un- 
derstanding of the auxiliary fields Xa in the saddle point 
equations H79I81|I . which are obviously equivalent to Eqs. 
(|54I56|) obtained from the two group Ansatz. As one may 
expect from the functional integral H76I) the variables Xa 
are prone to fluctuate most strongly along the softest 
mode of the TAP solutions, and this leads to a rather 
natural interpretation of the saddle point equations ]Ti)l 
1^ for a ^ b: Aab and Cab describe correlations between 
the soft mode directions and the local magnetizations, as 
also suggested by the two group Ansatz. 

For a = b extra terms arise. This is because Xa does 
not exactly describe soft mode directions in the same 
sense as nia describes magnetizations. While this is im- 
material for inter-state correlations, it introduces correc- 
tion terms when intra-state correlations are considered. 
In the next subsection, this will become clearer from a 
direct derivation of these contributions from generalized 
Ward identities. 



B. BRST symmetry breaking and the 
generalization of Ward identities. 

The action in the exponent of the integrand in Eq. 1(76(1 
is invariant under the fermionic BRST symmetry^ 

jji 

6mi = etpi, Sxi = -e—tpi, Sipi = -exi, Sipi = (83) 

If the dominant states are stable minima (cf. Sec. Ill A(l . 
the BRST-symmetry of the action ((76(1 is not broken by 
the saddle point Eq. (|77|l . and Ward identities 

{rriiXi) = (ipiipi), (84) 
(xiX^) = -TO {^pi^|Ji) , (85) 

hold. On the macroscopic level (upon average over sites 
i), these impose that the order parameters Aat = Cab — 
0. This can be appreciated by noting that Eqs. I(84I85() 
reproduce the diagonal saddle point equations ((80181(1 . 
considering that = 1 — Q- 

If instead the majority of states are marginally stable 
states (i.e., minima merging with saddles of rank one) 
the BRST symmetry is broken, translating into Aab 7^ 0, 
Cab 7^ 0. The Eqs. ((79181(1 result from a (macroscopic) 
generalization of the above Ward identities. One can de- 
rive them from direct inspection of the sum over TAP 
states with a regularization. As we did for the toy 



model of Sec. IIIII [Eq. I(26() ] we add a regularizing term 
eyip{Xx X {{mj})) to the weight in Eq. ifTSl) . As explained 
in App. O this procedure selects minimally stable states 
with a susceptibility diverging as 1/Xx as Ax 0: 

Zj{p;m,\x) = ^e-""'^^-^^(^™»>'+^^^(^™=>^ (86) 

Q = l 

The quantity X is an arbitrary, extensive, symmetric 
function of the average site magnetizations subject only 
to the constraint that its gradient be non-orthogonal to 
the soft mode of typical marginal states: 

VmX{m) -Sm^O. (87) 

This regularization explicitly lifts the marginality 
throughout the computation. Sending the control pa- 
rameter Xx eventually to zero allows us to recover the 
marginal states. 

When the TAP equations are perturbed by exter- 
nal flelds there is no guarantee that the number of 
solutions at a given value of the free energy density 
is conserved. Indeed only the set of TAP solutions 
counted by the BRST symmetric saddle point is robust 
to perturbationsiSi the number of solutions at any free 
energy density is conserved and there is a one-to-one cor- 
respondence between states before and after the pertur- 
bation. In the case of broken BRST symmetry the corre- 
spondence is lost and solutions can appear and disappear 
at any free energy density. This fragility is related to the 
marginality of the states. The regularization Eq. 1(86(1 
fixes this problem by lifting the marginality. As a conse- 
quence the number of selected states is conserved under 
perturbations (for fixed parameters / and Xx), but at 
the same time the BRST symmetry is explicitly broken. 

We will now derive the generalizations of the Ward 
identities 1(84185(1 . Moreover, by showing their equiva- 
lence with the self-consistency Eqs. ((79I81II . we shall 
confirm the interpretation of the order parameters of the 
two group Ansatz put forward in Sec. IIVBI 

We consider sums over TAP states which are dom- 
inated by shallow minima due to regularization. Fur- 
ther, we regard the states as functions of small perturb- 
ing fields {hk}, in the sense that mad/ifc}) denotes the 
unique solution of 9mi-FTAp({™j}) = hi in the vicinity 
of the unperturbed state ma{{hk = 0}). 

Since the free energy Hessian possesses a soft 
mode along the direction Srrii oc Q (normalized by 
N~^J2iCi — l)j the latter dominates the response to 
a perturbing external field. The regularization selects 
states whose susceptibility along the soft mode is finite, 
Xsoft = CiXijCj = (g^x)'^, where 5 is a selfaveraging 
constant, as shown in App. IO 

The coupling of perturbing external fields {hi} to the 
soft mode is given by their projection on the soft mode, 
J2iCihi- In particular, restricting to the response along 
the soft mode, we find 
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and thus, in the hmit of vanishing regularizer one finds a 
finite hmit for the derivative 



1 dXxX 
- lim — — — 




Smk (89) 



Note that the so defined vector {(5mj,} is proportional 
to the soft mode {(^k} and independent of X (since the 
constant g is proportional to X, and only the gradient of 
X in the direction of C matters). This same vector will 
appear again in the cavity method (see next section ^TJ. 
It gives a precise meaning to the amplitude of the soft 
mode appearing in the heuristic derivation of the two 
group Ansatz, cf., Eq. 

With the above observation, we are able to derive the 
following generalized Ward identities (see AppendixlBlfor 
the details): 

(m^^> = <5afc(Vr?-) + {<Sm^), (90) 
= -m6ab{i'lt) + {SmtSm1) (91) 



These equations are indeed equivalent to Eqs. (|80I81|) 
provided that we make the identification Aat — (iTiaSmb), 
Cab = {SmaSirib), as suggested by our interpretation of 
A and C as describing correlations among soft mode di- 
rections and magnetizations. 

In the absence of soft modes the limit Xx — > cancels 
the terms containing 6rn and the BRST-Ward identities 
(Eqs. H84l85|l ') are recovered automatically. 



VI. GENERALIZED CAVITY APPROACH AND 
THE TWO GROUP ANSATZ 

A. Cavity with marginal states revisited 

In the standard cavity approach one writes recursion 
relations for the local (cavity) field ho acting on an added 
site in terms of the local fields hi acting on its k neigh- 
boring sites when site is absent. Here, fc -I- 1 is the 
connectivity of the lattice. In Ising systems, the cavity 
field is a sum of messages 

k 

ho = ^Ui{h^;Joi), (92) 

4=1 

Ui{h,]Joi) = /3~Hanh~^[tanh(/3ft,,)tanh(/3Joj)],(93) 

where Joi are the quenched bonds coupling the cavity 
spin to its neighbors. The free energy gain for the 
addition of a spin at site is 



exp {-13 AF) = 2cosh{(3ho)Y[ 



cosh(/3Joi) 
^ cosh(/3ui) 



(94) 



Around the considered free energy density /, the den- 
sity of metastable states is assumed to grow exponen- 
tially as p{F) cx exp[TO(/)(F — Fo)] where m(/) = 



dT,{f)/df is the local slope of the complexity function 
(in other words, m is the Legendre conjugate of /), 
F — fN and Fo is an arbitrary reference value (to 
be absorbed into the normalization constant). In order 
to determine the average shift <I'cav(™) of this distribu- 
tion when a site is added, one averages the 'reweighting' 
factor exp{—(3mAF) over cavity iterations to obtain^ 
exp[-/3m$cav("i)] = (exp[-/3mAF])|^. j^j. 

In the case of the SK model the connectivity goes to 
infinity N = k + I —i- oo and the single bond strength 
tends to zero as Jq^ = 1/A^. The above relations thus 
simplify. In particular, we have Ui ~ JoinT-i with rrii = 
tanh(f3hi). The free energy shift due to a spin addition 
is 



exp(-/3Ai^) = 2cosh(/3ft,o)exp 
= 2 cosh(/3/io) exp 



i=l 



, (95) 



where (.) denotes a site average, and we have used the 
fact that in the large N limit the second term does not 
fiuctuate. 

If a marginal mode is present, this standard method 
fails since the addition of a spin has an anomalously 
strong impact on the system, rendering previous states 
unstable. In order to circumvent this difficulty, we regu- 
larize the problem once again by reweighting the states 
with a factor exp[XxX{{mi})], in addition to the stan- 
dard reweighting exp[— /3m_F({mi})] with respect to free 
energy which selects a certain free energy density. Even- 
tually, we will take Ax — > to recover the marginal 
states. As in earlier sections, the extensive observable 
X is an arbitrary symmetric function of the magnetiza- 
tions, subject to the requirement (|87|) . This method was 
originally introduced by Rizzo^ Here we go beyond his 
analysis by including the selection of a given free energy 
density, and establishing the precise connection with the 
two group Ansatz. Further, we provide a clear inter- 
pretation of the auxiliary cavity fields that need to be 
introduced. 

We expect from the toy models of Sec. IIIII that the 
regularization scheme will select minimally stable states 
with the lowest eigenvalue of their free energy Hessian 
being proportional to Ax (for a derivation in the present 
case see App. inj . The change of X upon addition of a site 
is most conveniently computed as a derivative of the free 
energy change with respect to the field hx conjugated to 
the observable X, 



AX = - 



dAF 



dh 



X 



(96) 



The regularization is implemented in the cavity approach 
by reweighting each cavity iteration by exp[AxAA']. 
Note that the exponent remains finite in the limit Ax — 
0, because the susceptibility of the soft mode diverges as 
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1/Ax- More precisely we have, 



dAF , dhi ^ dAF ^ 



where 



dhi dhx ^ dhi 

i=0 i=0 



Zi = j3 ^ hm A 



dhi 

Ax^o ^ dhx 



(97) 



(98) 



are finite fields proportional to the component Ahi of the 
local field fluctuation arising from a soft mode excitation. 
The fields Zi are in fact independent of the choice of X, 
as shown in App. lO 

Deriving the recursion relation H92f) with respect to hx 
we obtain a relation for zq 



zo 



dho 
dh~ 



N 



X 



^ Joj/3(1 - TOj)zj 



(99) 



1=1 

N 



^ Joi Srrii, 



i=l 



where we introduced the soft mode in the magnetizations 



6m, = = /3(1 - m2)z,. (100) 

dhi 

Note the correspondence with the definition (|89|) . which 
is most transparent if we notice that 



Srrii — [3 lim A 



drrii 



Ax-»o '^'dhx^ 



(101) 



using the definition H98|l . 

For the SK model, the shift of the regularizer, Eq. (|97|l . 
is readily evaluated using Eq. (|95|l 



N 



dui 



lim Ax AX = tanh(/?ft.o)/3zo - Vtanh(/3-u,)— -^/? 
\x^o ^-^ dhi 

i—l 



N 



1=1 

(3moZo - (3^ {rriiSmi) , 



(102) 



where mo = tanh/3ft,o- Similarly as in Eq. H95|l . we may 
neglect the fluctuations of the second term in the large 
N hmit. 



B. Free energy and self-consistency equations 

Putting elements together, we obtain the regu- 
larized free energy shift exp[— /3m<i>(,av] upon a site 
addition by averaging the two reweighting factors 



exp[— m/3AF] exp[AxAX] over all possible random con- 
figurations {/ii,Zj} of the neighboring cavity fields, as 
well as over the random couplings Joi 

exp[-/3m$cav] (103) 
= (exp(-/3mAi^) exp(AxAX)){ft,^^,,j,j. 

The first term has the standard interpretation of a 
shift of the exponential distribution of states. The second 
term equals 1 if the considered states are stable. In the 
marginal case, it captures information about the proba- 
bility for site additions to render existing states unstable 
or to make new marginal states emerge. 

We may use the central limit theorem to infer from 
Eqs. (|92I99I) that ho{{h,, z,, JoJ) and zo{{hi,z^, JoJ) are 
Gaussian variables with covariance matrix 



M 



Q* A* 
A* C* 



where 



^ N \ • 

Joi-rrii 

\i=l ) 



N 



N 



A* = Joirrii JoiSuij 



i=l 



= {midrrii) = (rni/3(l - mf)zi), 



N 



(104) 

(105) 
(106) 

(107) 



^4=1 



{{5m,f)^{[{l~mj)(3z,f), 



and (.) denote site averages. We can then reexpress 
Eq. flfel) as 



exp[— /3to$, 
— exp 
dhodzo 



m-(l 



27r[det7W]i/2 



exp 



D - 13^ A* 
2 -0 



(108) 
(109) 



[2 cosh(/3/io)]™ exp [tanh(/3ft.o)/3zo] 

with = (/loj^o)- The reweighting terms can alterna- 
tively be seen as describing the relative probability of a 
cavity configuration {hi, Zi, Joi} to occur, given a fixed 
free energy after the site addition. With this interpre- 
tation in mind, the probability distribution to find local 
fields ho and soft mode components zq on site can be 
read off from Eq. H108|) as 



P{ho,zo) = N ^exp 



2-0 



(110) 



(2cosh(/3/io))" exp [tanh(/3/io)/3zo] , 

where A/" is a normalization constant. The self- 
consistency of the cavity approach requires that the av- 
erage correlations on site are the same as on the neigh- 
boring sites, i.e.. 
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Q* = (to?) = j dhodzoP{ho,zo)ta.nh{pho)\ (111) 

A* = {niiSm,) (112) 

/■ , p., tanh(/3/io)/3zo 

J cosh (p/io) 

= (<5m?) (113) 

= JdhodzoPiho,zo){^—^) . 



Further, we confirmed once more the interpretation of 
the order parameters. 

To use the cavity method beyond the anneale d ap prox- 
imation is a rather cumbersome task, see Ref. [l3- It is 
much easier to carry out the two group computation with 
full replica symmetry breaking, even though it is less in- 
tuitive than the cavity approach. To help the reader 
appreciate the physical content of such a quenched cal- 
culation, we devote the following section formalism. 



Equivalence between the generalized cavity 
method and the two group Ansatz 



VII. QUENCHED TWO GROUP ANSATZ: 
FORMALISM AND INTERPRETATION 



The cavity approach with a single reweighting 
exp[— /?toA_F] corresponds to an annealed calculation, 
neglecting correlations and clustering among different 
states. Let us thus establish its connection with the an- 
nealed approximation in the two group formalism. 

It is straightforward to convince oneself that the above 
self-consistency conditions Eqs. H111I113|) are identical to 
the saddle point equations of the two-group Ansatz H50t 
I52|l . evaluated with the help of the annealed free energy 
expression (|57|l (see Appendix 1X1 for details). In partic- 
ular, we find that the two group order parameters Q, A 
and C coincide with the above Q* , A* and C*. 

We finally need to establish the precise correspondence 
between the regularized free energy shift $cav(™) com- 
puted within the generalized cavity method, and the 
replicated free energy density ^2G{m) computed within 
the annealed two-group replica Ansatz. We need to take 
into account that by adding a spin to an SK model with 
N spins, one obtains a system with slightly stronger cou- 
plings (by an average fraction of 1/2N) than in a stan- 
dard system with -1- 1 spins. This is basically equiv- 
alent to raising the inverse temperature to p (3' = 
(3{1 + 1/2N) simultaneously with the spin addition.^- 

We thus expect the relationship 



$cav(m) = $2G 



1 g(/3$2G) 

2 d(3 



(114) 



between the two free energy densities. Explicit evaluation 
of the righthand side using Eq. H47() indeed yields 



B d 

-/3m$2G- 2^(/3"^*2G) 



(115) 



l3^^+m,^{l-Q)-p^A, 



In Sec. IIVDI we have seen that for free energy den- 
sities f > f* the annealed solution of the two group 
replica approach is correct. For / < /*, however, 
the marginal metastable states are correlated and thus 
Qab, Aab, Cab 7^ for a ^ b. In analogy to the Parisi so- 
lution for the ground state of the SK model, we are look- 
ing for a hierarchical breaking of the replica symmetry 
encoding the assumption that marginal states are orga- 
nized in an ultrametric tree in phase space: the smaller 
the distance on the tree, the larger the similarity between 
the states. In particular, we expect the off-diagonal part 
of the matrices Qab, Aab and Cab to tend to the functions 
g(x), a{x) and c{x), respectively, describing the continu- 
ous breaking of replica symmetry. We continue to denote 
diagonal entries as Qaa = Q, Aaa = ^ and Caa = C. 



A. Replicated free energy 



With such an Ansatz for the overlap matrices, the 
quenched replicated free energy of m copies, Eq. H47|l. 
reads 



Pm<^Ta = /30(2: = O,y= (0,0)) 



(116) 



+ ^m(l-g) 



2 2 



dx q^{x) 



[2A + A^ + QC- 2(1 - m)AQ] 



+ — / dx [a^{x) + q{x)c{x) + 2m, a{x)q{x)\ , 
2 Jo 



which precisely coincides with —(3m^ca.v{'m) from 
Eq. (|108() . if we recall the annealed version of Eq. H57() 

for (j),n- 

We have thus proven the equivalence of the generalized 
cavity method and the two group replica calculation for 
all free energy densities in the annealed regime / > /*. 



where y = (2/1,2/2)- The function 0(a;,y) is the free en- 
ergy per replica of a subsystem of a; • m coupled repli- 
cas subject to external fields j/i,?/2 acting on the two 
groups of replicas (representing minima (-I-) and sad- 
dles (-)) with the same and opposite sign, respectively: 
y"^ = yi±y2/{2K) (cf., the analogous expression Eq. 



16 



for the magnetizations). More precisely, we define 
exp[x (bix,y)] = J2 exp[H(x,y,{sl})] , (117) 



=±1 



H(a:,y,K}) = ^ ^ ^ 4Q[x]^^,^^ 

a,b i.j 



a=l 



Here QHg'f, denotes the matrix Qab~Q'^''^' i^) restricted 
to a block of (xm) x (xm) replicas. 

The representation (|117|l allows for the derivation of 
recursion equations for <f>{x — dx,y) in terms of (/)(x,y) 
(see, e.g., Ref. 13). In the limit of continuous overlap 
functions {dx 0) they reduce to a Parisi's differential 
equation 



9^ 
dyf 



(3x 



dyi 



(118) 



dyidy2 



oyi oy2 



dyi 



where a dot denotes d/dx. The boundary condition at 
X = 1 follows from the definition (|117ll as 



0(a; = l,y) — lim —log / dhdz ^{h, z) 

K—foc p 



2 cosh /3 \^yi + h + 
2 cosh/3 [yi + h — 



y2 + z 

2K 

y2 + z 

2K 



(119) 

m+K 

-K 



7" log / d/idz /i(ft,, z) [2cosh/3 (j/i ^ 
P J 

X exp [(3{y2 + z) tanh/3(yi + h)] 



where 

/i(/i, z) dh dz 

A 



md\ 
d'^ 



27rVdet A 

Aq Aa 
Aa Ac 

{h,z). 



exp 



2 - 



(120) 

(121) 
(122) 



Here we introduced the notation Aq = Q — q{l) etc. for 
the jump between the diagonal entry and the closest off- 
diagonal elements of the overlaps. In general, these jumps 
are finite, indicating that individual marginal states are 
clearly distinct from their closest neighboring states in 
phase space. This is in contrast to the situation at 
/ = /eq where the overlap of neighboring states can come 
arbitrarily close to the self-overlap Q. 



B. Distribution of local fields 2/1,2/2 

Following Sommers and Dupontj^ we also introduce 
the distribution P{x,y) of local fields on the scale x re- 
quiring that 

«\ ■ • ■ O = / dyPix, y){{s:\ . . . 

for all X < {ai, . . . , a^} < 1. Note that here we need to 
keep track of the distribution of both fields j/i and 2/2 ■ 

In the continuous limit, the ensuing recursion relations 
relating P at x and x + dx lead to the flow equations 



(123) 



q 


'd^p 


- 2l3x 


d 


m 


2 




dyi 


c 


-d^p 


- 2Px 


d 




2 


dy2^ 


dy2 



d^p 



- (ix 



_d_rdi 

dyi \ dy2 



, dyidy2 

with the boundary condition at x = 
P(x-0,y) = 5(2)(y) 



dy2 \^ dyi 



(124) 



The joint distribution of local fields (t/1,2/2) within 
a typical marginal state (with free energy density / = 
/(to)) is eventually obtained from the convolution 



Pqu(y) = [2cosh(/32/i)]'"exp[/3y2tanh(/3yi)] (125) 



27rVdctA 



The variable transformation (|59I6U|I . m = tanh(/3yi) 
and dm = f3y2 / cosh^ (Pyi) , yields the joint distribution 
of local magnetizations and soft mode components, 

Pqu{m,Sm) = d'^yPqu{y)d{m-ta,iih{(3yi)) 



xS Sm 



Py2 



cosh {j3yi) 



, (126) 



one of the central objects of interest characterizing the 
metastable states. 



C. Self consistency equations for the order 
parameters 

Using the definitions of (p and P, one can convince 
oneself that the continuous limit of the off-diagonal self- 
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consistency equations (|50l52ll can be cast into the form 



q{x) 
a{x) 

c{x) 



d(f>{x,y) 
dy2 

<90(a;,y) / d<j>{x,y) 



dy2 



dyi 



^ d(j){x,y) 
dy2 



(d(t>{x,y) d(t){x,y) 
- m- 



V dyi dy2 
where we have introduced the notation 

/>oo poo 

[o(y)]^ = dyi dy2 P{x,y) o(y) 



(127) 

C 

(128) 
(129) 

(130) 



to denote an average over the local field distribution at 
the scale x. Alternatively, Eqs. (|127I129|I . can be derived 
from a variational formulation of the quenched prob- 
lem. 

For the diagonal (intra-state) overlaps (a = b) we have 



A = {rh Sm 
C = (Sm^), 



(131) 
(132) 
(133) 



where averages are over the joint distribution Eq. (|126|l . 
Integrating out explicitly the soft modes, this can be cast 
into a form similar to the annealed equations H66I68|I . 

Q = (tanh2(/3j/i)), (134) 

4^1 n n^^ (j/itanh(/3j/i)) 
A+l-Q = -Q—-mQ + , (135) 

C -2A-m(l-Q) = rr?Q + 2mQ^ + Q ( 

Aq \AqJ 



-2 

1 



Aa\ (?;itanh/3(2/i)) 



PAq 



(136) 



(3^Aq 



1 



Aq 



where (•••) denotes an average over Pqn{yi] 
J dy2Pqu{yi,y2)- 



VIII. INTERNAL CONSISTENCY AND 
THERMODYNAMIC STABILITY 



Sections IIVI IVl and IVII provide three equivalent meth- 
ods to capture the properties of marginal states at a given 
free energy density above /*, and the previous section ex- 
tended the formalism to the low energy regime. Yet, we 
did not bother so far about the internal consistency and 
thermodynamic stability of the obtained solutions. More- 
over, we merely stated the existence of marginal states 
without analyzing in more detail the eigenvalue spectrum 



of the free energy Hessian and the related question of lo- 
cal stability of the states. In this section we address these 
issues, and show how to obtain more information about 
the local environment of the marginal states. 

Even though the analysis of the local stability and the 
thermodynamic consistency are two a priori very differ- 
ent aspects of the problem, they turn out to be closely 
related. 



A. Internal consistency of the 2G solution 

In order to understand the local free energy land- 
scape of a given metastable state (TAP solution), we 
need to characterize the free energy Hessian of a typi- 
cal local minimum, or in other words, the inverse of the 
susceptibility matrix, Xij^ — /39i9j-Ftap({w})4S4i The 
fluctuation-dissipation relation requires that the ("zero- 
field cooled") susceptibility, i.e., the trace of x, be equal 
to /3(l — (5))ii However, not all solutions of the TAP equa- 
tions satisfy this constraint, but only those for which the 
inequality 



> 



(137) 



holds. Other solutions are unphysical. 

This condition has always to be checked separately, af- 
ter having obtained a self-consistent solution of site mag- 
netizations rrii. 

For iV — !■ cx), this can be rewritten as 



xp^l- (3^ {l-2q+ (m*)) > 



(138) 



where the average is over the appropriate magnetization 
distribution (Eq. Ht)5|l with rh = tanh(/3/i) in the annealed 
case, and Eq. H126() in the quenched case). 

In the regime f > f* the condition (|138|l is satisfied 
as a strict inequality in all marginal states. In contrast, 
if one aims at describing genuinely stable minima by im- 
posing saddle points that conserve the BRST symmetry, 
the condition (|138|l is always violated. This leads to the 
conclusion that in the SK model there are no genuinely 
stable TAP states which are not closely related to the 
family of dominating marginal states (see Ref. [23| [an- 
nealed case] and Refs. 29.4^ [quenched case]). While 
we know from the regularization procedure that there 
are actually stable states, they are always close to being 
marginal, sharing similar properties with the dominating 
marginal states. 

The only thermodynamically consistent states that do 
not break the BRST symmetry are the states at the equi- 
librium free energy density /oq for which the criterion 
(|138|l is marginally satisfied as an equality. 

We note that Eq. H138|) is actually equivalent to the re- 
quirement of a positive replicon A^, defined as the small- 
est eigenvalue characterizing the fluctuations of the repli- 
cated free energy as a function of the order parameter ma- 
trix Qab- In fact, Afi turns out to be simply proportional 
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to xpm^ The implications of a vanishing rephcon, and the 
possibiUty of its simultaneous occurrence with the break- 
ing of the BRST symmetry in the regime /eq < / < /* 
will be discussed below. 



B. The free energy Hessian 

The extensive part of the spectrum of the inverse sus- 
ceptibility matrix x~"'^({™l) & generic TAP state {rrii} 
was determined in Ref. j42|] from an analysis of the TAP 
equations, neglecting terms of order 0(1/ N). The exten- 
sive part starts off as a semicircle 



P(A) 



N 

71" VP 



v/A - Ao, A-Ao<l, (139) 



where the lower band edge Aq, together with the resolvent 
Tr[(Ao — X^^)^^]) follows from the solution of 



ro 
1 



-.fi{ro + Ao), 
f2{ro + Ao), 



fnix) 



1 ^ 
TV ^ 



(/f(T^+/3(l-0) + ^) 



(140) 
(141) 

w(142) 



The semicircle's amplitude is controlled hy p — fairo + 
Ao). The support of the continuous part was proven to 
be strictly positive)^ Ao > 0. However, this result, valid 
to leading order in N, does not exclude the presence of 
a sub-extensive number of negative eigenvalues. In fact, 
such regions of phase space must exist as guaranteed by 
the Morse theorem. 

One can easily check that if xp = 0, the solution of 
Eqs. P40I141|) is Ao = 0,ro = -/3(1 - Q), while Aq = 
implies xp = and ro — —/3(1 — Q)- In particular, the 
vanishing of the parameter xp and of the 'band gap' Ao 



occur simultaneouslytMii A, 



V(4p). 



At finite N the spectrum exhibits tails below the ex- 
tensive band edge Ao which may even extend to negative 
eigenvalues. However, as usually in random matrix prob- 
lems, it is reasonable to assume that their density decays 
exponentially as exp(— aiV-'^/^),^- so that they do not sur- 
vive for large N. 

However, there is one eigenvalue below the gap edge 
that survives in the thermodynamic limit. Indeed, in- 
cluding the corrections of order 0{1/N) in the analysis of 
TAP states, one finds that the Hessian possesses a single 
isolated eigenvalue, which is not captured by the analysis 
to leading order. Such an eigenvalue is a common feature 
of many mean-field spin-glass models. To our knowledge 
it was first encountered as the longitudinal eigenvalue in 
the spherical p-spin model^. where it is positive and the 
corresponding states are genuine minima. The situation 
is presumably similar in the free energy regime of the p- 
spin Ising model where states are stable. ^'^ In contrast, 
for the regime / > /* of the SK model it was proven that 



the isolated eigenvalue is exactly zero in the thermody- 
namic limit 1^ This provided the first evidence for the 
marginality of the dominant states. 

An exactly vanishing eigenvalue should actually be 
present in any BRST-breaking states. In this paper we 
used this insight as a starting point, assuming a soft 
eigenvalue Asoft, and deriving its self-consistency in vari- 
ous ways in the preceding sections. 



C. The spin glass susceptibility 



The spin-glass susceptibility is defined as 



XSG 



N 



— Try-^ 

N ^ 



-y- 



(143) 



where Xj are the eigenvalues of the Hessian. It has a 
simple expression in terms of the above introduced pa- 
rameter Xf— 



XSG 



1 — Xp 
Xp ' 



(144) 



which is valid for physical states with xp > 0. 

As ^ the spin glass susceptibility diverges. As 
mentioned above, this happens when the gap Aq vanishes, 
i.e., when there is an accumulation of eigenvalues of the 
Hessian at A = 0, cf. Eq. (|139() . Such states are "fully 
marginal" , in the sense that they possess an extensive 
number of soft modesi^ 

In marginal states with only one soft mode and a fi- 
nite gap Ao to the continuous part of the spectrum, the 
spin glass susceptibility remains finite. Indeed the linear 
susceptibility diverges only along the marginal direction 
which results in a non-extensive effect for xsG^ 
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FIG. 4: Spin-glass susceptibility of marginal states, xsG, 
versus free energy density / at different temperatures T = 
0.2, 0.1, .0.01, 0.001. The susceptibility is plotted in log-scale. 
The physical results holds for / > /* . /o is the point at which 
E goes to zero in the annealed approximation. 
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D. Discussion 

At this stage, we only have an exact description of 
the free energy regime / > /*, while an analysis of the 
low energy regime down to /oq requires the quenched 
solution of the two-group equations of Sec. IVIII The 
latter is a technically difficult task and will be carried 
out elsewhere S Here, we will content ourselves with a 
discussion of possible scenarios for the local environment 
of dominating states as one decreases the free energy. 

The spectrum of the Hessian of a TAP state depends 
on its free energy density /. As we have seen above, all 
states with free energies / > /* have a finite gap Aq > 
in the spectrum, while the thermodynamically dominant 
state at /cq is fully marginalji^ as implied by the van- 
ishing of the replicon, Aq oc oc A|j = 0. The latter 
is an interesting situation, which leads to a non-trivial 
dynamical behavior owing to the multitude of flat direc- 
tions in phase space. It is thus an important question to 
establish whether full marginality is a unique property 
of the equilibrium state, or whether there actually ex- 
ists a window of free energies foq < / < /gap where the 
dominant states are fully marginal. Here /gap denotes 
the highest free energy density where the gap Aq in the 
Hessian spectrum vanishes. 

The following scenarios are possible in principle: 

• Scenario I (see Fig. |S1 top). In the simplest sce- 
nario, the gap Aq tends smoothly to zero as / tends 
to /eq (together with the BRST breaking order pa- 
rameters A and C) and thus /gap = /oq- While 
the soft mode tends to become orthogonal to the 
magnetization (A — > 0) its amplitude vanishes si- 
multaneously (C — > 0) as the equilibrium state is 
approached. Under this scenario, only the states 
at /oq are fully marginal and the formalism devel- 
oped in Sees. IIVIVIII should provide a consistent 
description for the dominant states (with a finite 
complexity) at all free energies and temperatures. 

• Scenario II (see Fig. [SJ bottom). Fully marginal 
states already occur at /gap > /oq. In this case, it 
is a priori not clear whether the notion of the (iso- 
lated) soft eigenvalue continues to make sense, since 
it plunges into a continuum of other eigenvalues. 

If, nevertheless, one of the many soft modes can be 
singled out (e.g., with the help of the projector term 
appearing to order 1/A^ in the TAP equations^i^), 
the two group formalism may probably still de- 
scribe this regime. Conversely, if one finds a ther- 
modynamically consistent two group solution with 
Ao = 0, this strongly suggests that the singled out 
soft mode still preserves a meaning. In this case, 
the solution would still be characterized by bro- 
ken BRST symmetry (C > 0), but we conjecture a 
vanishing order parameter A = 0: while the mag- 
nitude of the soft mode stays finite, its direction 
is expected to be orthogonal to the magnetization 



vector in phase space, as we will argue in the next 
section. Further, we would expect that this branch 
of solutions continuously joins the BRST symmet- 
ric Parisi solution, (i.e., C ^ 0) as / ^ /oq. Such a 
branch would yield an extensive complexity, imply- 
ing an exponential number of fully marginal states. 

On the other hand, the isolated soft mode might 
lose its meaning as the gap closes, and the two 
group Ansatz might have no solution with non-zero 
soft mode amplitude C, suggesting that both A and 
C vanish at /gap- With A = C = 0, however, the 
description of the system reduces to the standard 
Parisi ("one group" - BRST symmetric) Ansatz, 
and it has been shown in earlier studies^Siii, that 
no thermodynamically consistent BRST symmetric 
continuation of the equilibrium state at /eq exists, 
not even within a quenched RSB computation. The 
only option left would then be the possibility that 
no states at all exist in the interval ]/cq, /gap [, unless 
a completely different replica symmetry breaking 
scheme is considered. This last scenario is rather 
unlikely, given that the absence of states between 
/oq and /gap is hardly reconcilable with numerical 
studies of TAP solutions^ which provide evidence 
for TAP states basically at all free energies. 



I 




FIG. 5; Possible scenarios for the gap in the spectrum, Ao, 
at free energy densities below the threshold of validity of the 
annealed approximation, /*. Scenario I: the gap Ao goes to 
zero at feq, as well as the soft mode overlaps {a{x),A} and 
{c{x),C}. Scenario 2: the gap goes to zero at higher free 
energy (/gap) implying the existence of fully marginal states 
also above the global minimum. 



20 



IX. LOCAL FIELD DISTRIBUTION IN THE 
ANNEALED REGIME 

In this section we analyze in more detail the local field 
distribution Pann(^), Eq- (|^ - of marginal states in the 
free energy regime / > /* where the annealed descrip- 
tion applies. We remind the reader that the distribution 
depends on the selected free energy density via the Leg- 
endre transform parameter m. 



A. Low temperature analysis 

The low temperat ure limit of the complexity was stud- 
ied long ago in Refs. ll8l54l55i Here, we focus on the sus- 
ceptibility and relate it to the properties of P{h) which 
are rather unexpected and, to our knowledge, have not 
been discussed before. 

As usual, the relevant range of the Legendre parameter 
m scales as m T at low temperatures. We thus use the 
variable ui = m/T to obtain a sensible T = limit. We 
anticipate that the susceptibility behaves as x = — 
Q) ^ T, similarly to what is known from Parisi's ground 
state solution. This assumption will turn out to be self- 
consistent. The saddle point equations (|67l()8|) for the 
order parameters A, C suggest a low temperature scaling 



lim f3A= lim = -e - ^, (145) 

/9 — >oo (3 — >oo 2 2 



where 



lim 



2/3- 



= lim 

2 (3^oc 



— oo 
oo 



dh Pann(^) h tanh(/3ft,) 

dh Pa„n(/l) \h\ (146) 



is the (w-dependent) energy density of the selected states, 
as follows from the low temperature limit of the energy 
E = SiJijSj and the local fields hi = J2j JijSj- 

In the regime of local fields 



riog(l/r)</i</5A 



(147) 



the field distribution Eq. takes the low temperature 
form 



-Pann(/l) = A^Q ^ ^Xp 

Afo = / dhexp 



\h\ 



\h\+er 



\h\ 



2' ' 2 
The self-consistency Eq. (|67|) then reduces to^ 

e = /" dh \h\ P,n^{h) 

e uj exp[— e^/2] 
2~4 J[q ' 

which relates e and the Legendre parameter w. 



(148) 
(149) 

(150) 



B. Soft mode direction 

As a corollary of the low T scaling Eq. H145() we obtain 
the behavior of the angle between the soft mode and the 
magnetization as a function of (free) energy density: 

7(e;T)«|-[-e-c.(e)/2]i/2TV2, e > (151) 

where the T = limit of the annealed threshold /* is 
e* = —0.672. In particular, we note that the lower the 
temperature the more the soft mode tends to be perpen- 
dicular to the magnetization, cf. Fig. O 



C. Susceptibility 

In order to confirm that indeed x ~ T we write 



x = /3(i-Q) 



dhPi^nnih) 



cosh^ {(3h)' 



(152) 



The integral is dominated hyT-^h^ Tlog(l/r), where 
the local field distribution simplifies to 



Pann(ft-) ~ A^Q ^ ^^p 



/?(-£- c^/2) 



(153) 



cosh^ {(3h) 2 
Changing variables to ^ = /3/ cosh^(/3/i), we indeed find 



X 



ff^ , exph(e-tj/2)£-e2/2l T , , 



where we have made use of Eq. (|150ll . Surprisingly, the 
susceptibility does not depend on the energy density e in 
the low T limit. The origin of its linear T dependence 
will be discussed further below. 



D. Discussion 

1. Pseudogap and full marginality 

The local field distribution Eq. Ij 153(1 is rather peculiar 
(it is plotted for T = 0.01 in Fig. E}. It differs signifi- 
cantly from that in the equilibrium state, both for ther- 
mally active sites (with fields of order /i ~ T) as for larger 
fields of order h ^ 0{1). 

The field distribution of the ground state is known to 
assume a universal low temperature scaling form^ 



P{h,T) 



Tiljih/T) for /i - T, 
est. \h\ for > T, 



(155) 



which is related to an asymptotic fixed point in Parisi's 
fiow equationsi^ The expression (|155ll implies a suscep- 
tibility linear in T, arising from 0(T^) spins with a local 
linear response of order 1/T. 

In contrast, in the marginal states at higher energies, 
f > f*, the linear susceptibility, Eq. H154|) . is due to 0{T) 
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spins with a local response of order 0(1) (on sites with 
fields h ~ Tlog(l/T)). On the other hand, the density of 
really active spins (subject to local fields of order h <T) 
is exponentially suppressed at low temperature, 



Pann(/i - 0) - exp[-/3(-e - Lo/2)]. 



(156) 



The scaling behavior Eq. \ibb\ in the ground state is 
closely related to its full marginality.^^-*^ Indeed, full 
marginality requires the vanishing of the replicon - see 
Sec. IVIII Bl (A B cx — !■ 0), which in turn is equivalent 
to the marginality condition 



1 = (3^ dh 



-Pqu(fe) 

"cosh*(/3/i)' 



(157) 



Here, the local field distribution Pqu{h) is given by the 
quenched Parisi solution (i.e., Eq. H125|) in the limit A ~ 
a(x) = C = c{x) = Aq = 0, with yi = h and 2/2 being 
trivially integrated out). From the equality H157|) . valid 
for all T, one infers the low temperature scaling (|155|l . 

With the connection between full marginality and h/T- 
scaling in mind, the lack of the latter in higher marginal 
states (with only one soft mode) may not be too sur- 
prising, since we know that above /* the states are not 
fully marginal (the replicon remains finite through- 
out). Further consequences for the regime h — 0(1), 
such as the absence of the linear pseudogap implied by 
Eq. (|155|l . will be discussed in the next subsection. 

There is a noteworthy aspect of the nearly hard gap 
in the low field distribution Eq. H156() : the exponential 
suppression of Pann(^ ^ T) (see inset of Fig. ^ decreases 
with —e — uj/2 « f3A and eventually would vanish at low 
enough energies as A 0. This is, however, preempted 
by the breakdown of the annealed approximation and 
the occurrence of full replica symmetry breaking at /* 
which requires a quenched computation. Nevertheless, 
the above observation suggests that the vanishing of A 
is connected with the onset of full marginality, i.e., the 
approach to zero of the continuous spectrum of the free 
energy Hessian. This is supported by a result by Parisi 
and Rizzo^ who showed that as the continuous spectrum 
approaches Xq 0, the (isolated) marginal mode be- 
comes orthogonal to the magnetization (that is, A = in 
our formalism). We thus conjecture that in general the 
vanishing of the gap in the spectrum, Aq oc = 0, im- 
plies the vanishing of Aab- With respect to the discussion 
of the previous Sec. IVIII Dl this is guaranteed to happen 
in scenario I, while it constitutes a non-trivial prediction 
for scenario II. 



2. On stability and slow dynamics 

The field distribution Eq. H153|) differs from the equi- 
librium one also in the range of fields h ^ 0{1). In the 
limit T —^ 0, the fraction of fields with a fixed magnitude 
T <C ^ 1 remains bounded from below by a finite con- 
stant, cf. Eq. lt)5|) . In particular, there is no linear pseu- 
dogap P{h) ^ \h\, in contrast to the equilibrium state, cf. 




FIG. 6: Local field distribution Eq. 115311 in the annealed 
regime for /max and f* at T — 0.01. In the inset the region 
around h = is enlarged to show the nearly hard gap occur- 
ring at any / in the annealed regime. The vertical line marks 
TlogT which is the scale /igap at which the gap is expected 
to open. 



Eq. H155() . Strictly at T = there is a finite probability 
of configurations dA, h — 0, while at finite T there is an 
almost hard gap on the scale T log T as discussed in the 
previous section. 

The absence of a linear pseudogap immediately raises 
the question about the stability of these states. As was 
realized in the early days of the SK model, a linear pseu- 
dogap is the minimal suppression of the low field distri- 
bution for a truly stable state (see, e.g., the discussion in 
Ref. [5^ ]. a configuration with a finite density of local 
fields around h = being unstable to the flipping of a 
finite number of spins (exactly at T = 0) . A very similar 
argument lead Efros and Shklovskii to infer the presence 
of the Coulomb gap in long range interacting electron 
glassestfifi Recently, it was recognized that both pseudo- 
gaps are related to the full marginality of the equilibrium 
statei^a 

The absence of a pseudogap in marginal states would 
seem to render them unstable with respect to a finite 
number of spin flips. While this is true strictly at 
T = 0,"® the finite temperature analysis is more sub- 
tle, because the thermodynamic limit and the T ^ 
limit do not commute. One can ap ply stability argu- 
ments analogous to those of Ref. j53| to the field distri- 
bution Eq. H153|) of marginal states above /*. Such an 
analysis shows that at low but finite temperature a collec- 
tive flip of order 0([Tlog(l/T)]^A'') spins (randomly cho- 
sen among the sites with small local fields) is necessary 
in general to render the state unstable. This contrasts 
with 0{T^N) flips for states with a linear pseudogap, 
but no hard gap on the scale oi h ^ T. The logarithmic 
enhancement of stability against random spin flips for 
/ > /* is indeed due to the presence of an almost hard 
gap on the scale of /igap ~ riog(l/T), see inset of Fig. 
One may therefore expect that a finite T dynamics based 
on random activated spin flips exhibits similarly long (if 
not longer) escape times as low energy states with linear 
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FIG. 7: Overlap Q versus angle 7 a T = 0.1. The soft mode 
becomes more and more orthogonal to the magnetization vec- 
tor as the free energy / and the Edwards- Anderson parameter 
Q decrease. The vertical line indicates 7 = tt/2, which is not 
reached within the annealed regime. The dashed low energy 
part of the curve (3(7) is unphysical in the annealed approx- 
imation. 
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FIG. 8: Free energy dependence of the order parameter Q 
in the two group approach at T = 0.1. Note the decrease 
of Q{f) with free energy. For comparison, we also plot the 
(unphysical) BRST-symmetric solution where Q{f) displays 
the opposite behavior, /max denotes the value of maximum 
E, fo and /hbc the point of vanishing E (lower band edge 
and upper band edge, respectively). Above /* the annealed 
solution is exact. 



pseudogap. 

On the other hand it is clear that a collective flip 
of a set of spins with a significant projection onto the 
marginal mode will take the system immediately out of 
the local state since the free energy barrier presumably 
decreases to zero in the thermodynamic limit as sug- 
gested by recent simulations.'^^ However, such an escape 
may be very difficult to realize since it amounts to flip- 
ping a large number of spins in a concerted manner. In 
summary, the presence of a single marginal direction does 
not seem to decrease the metastability of higher-lying 
marginal states in a dramatic way, rather their stability 
seems comparable to that of states closer to equilibrium. 

At the present stage the question as to the conse- 
quences of marginal states and their local environment 
on the dynamics is still open. Nevertheless, assuming 
that the dynamics is eventually dominated by the 'easy' 
escape via the marginal mode, we may expect that the 
rate at which the Edwards-Anderson parameter Q de- 
creases depends crucially on the angle 7 between magne- 
tization and soft mode, cf., Eq. H39|) and slows down as 
003(7) = A/^/CQ with decreasing /. This is illus- 
trated in Fig. [71 where we plot the decreasing overlap Q 
as a function of the increasing angle 7. 

This picture is consistent with relaxation dynamics 
that takes the system towards lower and lower lying 
metastable states. Indeed, Q{f) decreases as / decreases 
(at least in the annealed regime, see Fig. ISJ. At first 
sight, this result looks counterintuitive, being opposite 
to the behavior of the self-overlap in familiar one step 
glasses with genuinely stable states (e.g., p-spin models 
below the marginality threshold) where Q{f) increases as 
one descends to lower energies. It may be interesting to 
note that an increase of Q{f) with decreasing / is found 
in the BRST-symmetric solution of Eqs. (|54I56(I . see in- 
set of Fig. |S1 However, in the SK model this solution is 



thcrmodynamically inconsistent at all free energies above 
/oq, even within a quenched computation,^- 

The above observation suggests to think of the 
marginal metastable states discussed here as relatively 
small pockets of phase space that restrict the local mag- 
netizations to slightly higher values than in more equili- 
brated states. Relaxation dynamics out of these shallow 
traps, and subsequent descent in free energy will allow 
the spins to explore more degrees of freedom and to lower 
the self-overlap Q. Clearly, it will be important to check 
such a picture in analytic studies of glassy dynamics as 
well as in simulations, for which the interpretation of the 
order parameters in terms of soft mode correlations may 
provide helpful guidelines. 

X. OUTLOOK AND OPEN QUESTIONS 

Our study of the local landscape around a given 
metastable state leaves open the question as to the orga- 
nization of these states within phase space. In particular, 
one would like to understand the typical fate of a system 
once it manages to escape from a local marginal trap via 
the soft mode. This would be an important element to a 
more complete understanding of relaxation dynamics as 
well as to avalanche-like dynamics observed when the SK 
glass is driven with an external field^ 

The toy model of Section ITTll was very helpful to un- 
derstand the meaning of the two group Ansatz. Similar 
models may be guidelines as to how to use 'vectorial sym- 
metry breaking' in order to obtain physical information 
in complicated glasses, and to address questions such as 
the above. A natural example is the random field Ising 
model, where an extension of the two group Ansatz to 
three groups can be used to describe dominant Griffith 
singularities, and to capture rare disorder configurations 
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which describe the emergence of metastable states in the 
ferromagnetic regime^ 



A. Describing barriers with replicas 

Another interesting appUcation of the two group 
Ansatz might be the study of barriers. Indeed, from the 
solution of the toy model it is clear that replica saddle 
points with finite k2 = K describe disorder realizations 
with two stable minima separated by a finite barrier of 
order 0{1/K). It may be interesting to pursue this idea 
and to work at fixed ii" < oo in order to impose a cer- 
tain barrier height, similarly as fixing m allows one to 
choose the mean free energy density of the minimum- 
saddle pairs. One may hope that such an approach will 
yield information on the properties of barriers between 
metastable states, which is a notoriously difficult and un- 
resolved problem in glasses. 



B. Properties of states in the correlated regime 

Even though we are able to describe some key features 
of metastable states in the high energy regime / > /* 
where most marginal states are uncorrelated among each 
other, the analysis of lower-lying states in the free en- 
ergy range /cq < / < /* requires full replica symmetry 
breaking on top of a two-group replica structure. The 
assumption that marginal states are clustered in a hier- 
archical manner in much the same way as states close 
to the ground state energy suggests to look for a Parisi- 
type Ansatz in this energy range, the physical content of 
which we have described in Sec. IVIII 

It will be important to establish how the characteris- 
tics of the local landscape (marginal mode, gap in the 
spectrum of the Hessian) behave as a function of free en- 
ergy in this regime. In particular, one would like to know 
whether the breaking of BRST-symmetry persists in the 
whole interval, as actually suggested by the impossibility 
to construct a physical BRST-symmetric solution at any 
/ > fgq^SJ^ This issue is closely related to the inter- 
esting open question: do any typical states with / > /cq 
display full marginality, or is this intriguing property (im- 
plying a divergent spin glass susceptibility, critical fluctu- 
ations and dynamics) unique to the ground state? This 
important issue will be clarified by the quenched solution 
of the two group Ansatz iS 

It will be interesting to extend the two group analysis 
to other mean- field like models, such as random mani- 
folds with a large number of transverse dimensions^ or 
electron glassesi^2iS£ In the latter problem, the presented 
techniques may help to address experimentally relevant 
questions concerning the evolution of the Coulomb gap 
with decreasing free energy. 



C. Ground state properties in general mean field 
glasses 

The SK model is rather special in that its equilibrium 
state is always fully marginal below the glass transition. 
This observation, and the above discussion of the ap- 
proach to full marginality as a function of free energy 
raises a question about the nature of low energy states 
in more general mean field glasses. In the Ising p-spin 
model it has been established^ that above the so-called 
Gardner temperature Tq the states in an energy interval 
[/cq, fa] including the ground state are stable minima 
described by (one group) one-step RSB. At higher en- 
ergies, marginal minima-saddle pairs dominate, like in 
the SK model (two-group one-step RSB). Precisely at 
/ = /g where the two regimes meet, the states are fully 
marginal. At T = Tq the energy interval of stable states 
shrinks to zero, fa /oq, and consequently the (one 
step) ground state displays an instability. What happens 
to the ground state at lower temperature is not known. 
Both the scenario of a permanently fully marginal ground 
state similarly to the SK model, or a marginal ground 
state with a single soft mode (continuing the branch of 
solutions at higher free energies above Tq) can be imag- 
ined. This open problem is currently under investigation. 
Similar questions arise at phase transitions in many opti- 
mization problems (at T = 0) on diluted lattices. In this 
context, it would be very interesting to investigate the 
physical meaning of soft modes at zero temperature^ 



XI. CONCLUSION 

In this paper we have discussed three equivalent ap- 
proaches to the description of metastable states in mean 
field glasses - the replica two group formalism, the count- 
ing of TAP solutions, and the cavity method. We have fo- 
cused on the generalizations of these techniques allowing 
one to capture the marginal states that generically domi- 
nate the free energy landscape at high enough free energy 
densities, and probably play an important role in the dy- 
namics of these glasses. We found the physical meaning 
of the additional order parameters (generalized overlaps) 
arising in the marginal case: they describe correlations 
and relative orientations between the magnetization and 
the soft mode of marginal states. 

We show the two-group replica formalism to be an ef- 
fective and compact computational tool to obtain most 
of the key features characterizing the local landscape of 
marginal states (such as the distribution of local fields, 
the spectrum of the free energy Hessian, the direction 
of the soft mode, etc.), in particular when it comes to 
the description of correlated low energy states. We have 
made an effort to exhibit the physical content of this 
Ansatz in order to render this tool accessible to a wider 
audience. Extensions of this Ansatz can be of use in 
other problems, e.g., for the study of barriers in glasses. 
Simple toy models as introduced in the beginning (Sec. 
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mill may serve as useful guides in the development and 
interpretation of such techniques. 

Revisiting the uncorrelated high energy regime of 
metastable states where an annealed disorder average can 
be performed, we have discussed their properties in the 
light of the new interpretation of order parameters. Fur- 
ther investigations will be required to establish the dy- 
namical significance of these states. 
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APPENDIX A: EXPRESSIONS FOR (/>,„ 

The log-trace term (|48ll in — /3to$2G is given by 

Here we derive two representations of this term, (Eq. H53|) 
and Eq. H57() '). in order to facilitate the connection with 
the counting of TAP states of Sec.lvland the generalized 
cavity approach of Sec. IVII respectively. 

Decoupling spin products in Eq. IjAljl by means of 
Hubbard-Stratonovich transformations, one obtains the 
representation 



if" 



/ det r+ det T- 



detQ 



dga dg+ dg^ 
\/27r ^/27^ V27r 

a—1 V V V 
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^exp 
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E /3(5a+.9+)^;+ 

m 

+ E f^i9a+9aya- 



(Al) 



X exp 



with n X n matrices 



T; (9aQab9b + ga^+\b9b + 9a ^ -\b9b 



r±,af) = ±- 



Cab 



K 2K^ 



(A2) 



1. Saddle point for for large K. 

For large K one can expand the F matrices in Eq. (|A2I 



K 



R = --A-^CA-^. 



(A3) 
(A4) 



We isolate the terms of order 0{K) in the exponential 
of Eq. IjAljl . and perform the spin sums, 



_ \K\^ r A dga dg+ dg- 

I det A| VHetg J 

exp< m ^ In 2 cosh[/3(ga + .9a )] 
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cosh[(3{ga+gi)] 



(A5) 



-K 



Vln 

^ cosh[P{ga + ga)] 



-E 

2 ^ 
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Abi9a9b - 9a9b ) 



9aQab9b + Rab {9a 9b + 9a 9b 



In the limit K —^ oo we can take the saddle point approx- 
imation for the term in the exponent proportional to K 
which leads to the saddle point equations (with respect 
to 9t,ga) 

tanh[/3(5a+.g+)] = -f5-'A-,'g+, (A6) 
tanh[/3(ga+g-)] = -r'A-.'g^, (A7) 

determining the location of the saddle point as g^ — 
9a ~ 9ai{9c})'i 9t g^ can then be integrated out. 
Decoupling the term gaQab9b with a further Hubbard 
Stratonovich field Xa^ and changing variables from to 



rua = ~f3 ^Ajgl{{ga}), 
one eventually obtains the expression [Eq. H53|) ]. 
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Note that the sum over (a, b) also includes diagonal terms 
with a = b. The Gaussian integral over Xa can be carried 
out in principle. For the annealed solution this leads to 
an expression as in Eq. ()64|l upon a change of variables 
TOa = tanh(/3/ia). 



2. Cumulant expansion in Qa- 

Alternatively, we may proceed from Eq. (|A1|I by chang- 
ing integration variables to 



9t +9a 



ha = 9a + 



9a + 9a 



Za = K{g+ ~g^ ). 



(AlO) 

(All) 
(A12) 
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The fields acting on the spins in Eq. ljAl|l turn into ga + 
= ha i: Za/2K, while only occurs in the Gaussian 
weight and can be integrated out. The log-trace term 
then takes the simpler form 



VdetM 
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27r 
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X exp 
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where ^ = (ha, Za), a = 1, . . . , n. The covariance matrix 
is a 2n X 2n matrix given by 



M = 



n n 



f Q A \ 

V A C )■ 



(A14) 



i.e., a 2 X 2 matrix (as in the annealed case where off 
diagonal terms vanish) with n x n matrices as entries. 
Its determinant is that of D = QC — A^, and its inverse 
is 



1-*^^ ] ab ^ ^ ] ac 



Ccb ^Ach 
~Acb Qcb 



(A15) 



Carrying out the spin sums and taking the limit K 
oo, one finds the contribution to the replicated free en- 
ergy [Eq. EZJ] 
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exp < ^ [mlog2cosh(/?/ia) + tanh(/?/za)^Za] 
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The integration over Za in the annealed version of 
Eq. HA16P leads again to the expression Eq. 



3. Selfconsistency equations and equivalence 
between two group and cavity approach 

With the help of the measure in Eq. PT3^ . it IS easy 
to see that every spin s*" under the averages in the two- 
group self-consistency equations, Eqs. H5UI52|I . is replaced 



by tanh(/3(/ia -I- a Za/2K)). In particular, in the limit 
iiT — > oo we have 

s-a' ^ tanh(/3(/i, + a Za/2K)) (A17) 
tanh(/3/ia) = rha, 

K [tanh(/3(/i, -I- Za/2K)) ~ tanh(/3(/i„ - Za/2K)))] 

K^ao l3Za 



C0Sh^(/3 /la) 



Snia, 



(A18) 



where we introduced the variables rha and Sma- This 
leads to the self-consistency equations (|61I63|I . and es- 
tablishes the equivalence with the expressions from the 
cavity approach, Eqs. H111I113|I . on the annealed level. 



APPENDIX B: GENERALIZATION OF WARD 
IDENTITIES 

In this appendix we derive generalizations of the BRST 
Ward identities, reproducing the saddle point equations 
for marginal states. We consider sums over TAP states 
such as in Eq. JHSJ. We can assume the states to be min- 
ima (due to regularization), which we regard as functions 
of external fields {hk}, in the sense that ma{{hk}) is the 
unique solution of 9i-FTAp({wj}) = hi in the vicinity of 
the unperturbed state ma{{hi = 0}). 

After replicating the system n times (with n ~* even- 
tually), we consider the identity 



P{m1xl)Z'} = ^ _{<^^exp5][AxA({m^=,J) 

a— 1 ^ c— 1 

-/3mi^TAp({m^=,J)]}, (Bl) 

where the limit Ax will be taken at the end. The 
function Zj is the TAP partition sum which can be ex- 
pressed both as the functional integral Eq. (|76|l , including 
the regularizing term XxX{{nii\) in the exponential, or 
as the sum over TAP states Eq. . The average on the 
left hand side is taken over the measure defined by the 
action Eq. {TSJ. Evaluating the right hand side we find 



(B2) 



E 



dhl 



(3mm 



a QFtap 



dhi 



{ \ dhi 

X exp [-PmFTAp{{mlJ) + XxX{{mlJ)] | 
dhi 

where the stationarity of -Ftap was used. Using the defi- 
nition of the soft mode, Eq. (|89|) . and summing Eq. ljB2|) 
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over i = k, we establish the generahzation of the first 
Ward identity [Eq. P7|)]. 

/3(m°a;'') = Sabi'^ai^) + PimaSmt) . (B3) 

The second identity follows similarly from the relation 
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(B4) 



dhfdh'' 

■I, a '■ ^ 



X exp I ^[-/3mFTAp({m^^, J) + AxX({m^^_ J)] 

U=i 

-- -0^m5ab ii^ai^b) + P'^ {SrUaSmb) 



TV 

The very last term can be rewritten as 



dhfdK 



1 V — 



Xx 
N 



SabJ2 



(B5) 



i.k 



i,k,l 



dm'^drrti 



a ^ki 



i,k 



dX dml ^ 

-m 

dml dhf ' 



1 



where we have used the identity 



Ax TV' 



Ax TV' 



X 



dhidhi 



dhk 



^2f3m,^, (B6) 
ohi 



and the definition Eq. H89|l for the soft mode (5m. Fur- 
thermore, a spectral decomposition of the susceptibility 



matrix x (a-s in Sec. IVIII|) shows that the last term 
of Eq. (|B5p . N'^Tilxd^Xx], consists of a contribution 
(AxTV)^^ due to the soft mode, and a further contribu- 
tion ~ Ax- Both are negligible, as the limit TV —> 00 is 
taken before Ax 0. 

Finally, inserting Eq. (|B5p into Eq. (|B4|I we find 
Eq. 



APPENDIX C: EFFECT OF REGULARIZATION 
IN THE PRESENCE OF MANY STATES 

By introducing a regularizing weight factor exp(Ax-'^) 
into the cavity formalism and the explicit sum over 
TAP solutions we shift the weight slightly towards sta- 
ble states. The following constructive description of the 
selected set of states may be helpful to understand this 
formal trick even though we do not have a rigorous proof 
for its correctness. 

The selected states will be slightly stable. However, 
under a small rise of the temperature they would be- 
come marginal again. We assume that the states are 
selected by a weight function X affecting all states in the 
same (self-averaging) manner (with X a symmetric func- 
tion of the TOi, e.g., X = X]iV'("T'i) with arbitrary 
Then most of the selected states will become marginal 
at the same slightly higher temperature Tx = T + ST^. 
Likewise, almost all states that are marginal at Tx will 
adiabatically evolve into typical states selected by XX at 
the lower temperature T. We my thus expect the num- 
ber of selected states to be given by exp[NY^{T\, f\)], 
where f\ is unequivocally determined by the considered 
free energy density / at T. In general, one expects the 
complexity to decrease proportionally to the increment 
of temperature, 



(Cl) 



The properties of the selected states at T follow from 
perturbation theory around the marginal situation at 
(7a, /a)- One finds that the temperature shift dT\ in- 
duces a change of local magnetizations of order Ami ^ 
{STxY^^d along the soft mode {Ci}, the square root re- 
flecting the anomalous response of the soft mode. Ac- 
cordingly, the value of the regularizer changes by SX — 
VX-Srh ~ N{5T\Y^'^, and the soft mode acquires a finite 
susceptibility x^^k = E., Gx.^'O {ST^f'^. 

The weighting procedure favors the states optimiz- 
ing exp[TVE(TA,/A) - X5X], which yields {STxf' ^ A, 
very similarly to the mechanism in the toy model of 
Sec. mil Consequently, the susceptibility along the soft 
mode scales as 



1 



1 



Xsoft 



(<5rA)i/2 A- 



(C2) 



Note that all proportionality constants in the above argu- 
ments are self-averaging constants. This implies in par- 
ticular that g = (xsoftA)"^ is independent of the specific 
metastable state. 
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